Moz not crawling opencart product pages
-
hi,
i have waited for over 2 weeks now and the crawler only got 8 pages, and is not getting all the open cart pages and products.
any idea of what can be wrong?
im using joomla 2.5.11 and mijoshop 2.0.5 (which uses opencart 1.5.5.1).
thanks
-
Hi Marlon,
There are a lot of different issues that may cause Rogerbot to be unable to crawl your site. For troubleshooting tips, I suggest reading http://moz.com/help/pro/why-can-t-rogerbot-crawl-my-site. Hope that helps!
-
ok thanks
-
Screaming Frog has an option in the menu -
Configuration > Spider > "Ignore Robots.txt"
Tick this box and re-scan and see if you get different results.
And also, may be worth checking this out:
http://moz.com/learn/seo/robotstxtGood luck!
-
hi
thanks for the answer, tried screaming frog now as well and seems it has the same problem,
is it possible to allow only certain spiders/robots to crawl all files, with the robots.txt?
thanks
-
It's hard to say without knowing the domain name -
Have you tried Screaming Frog software to see if it also has issues crawling?
Double check your robots.txt to see if some directories and pages are blocked.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How does Moz gather the Keyword for my campaign in Moz Pro?
Looking at my campaign on Moz Pro and I was wondering where the keywords are generated from as I did not put these in. Can someone please explain?
Moz Bar | | House-Doctors0 -
How do I pull "Mobile Rankings" in MOZ?
My client wants me to pull mobile-specific rankings through Moz's keyword explorer. Does anyone know if this is possible? Thanks!
Moz Bar | | TaylorAtVelox0 -
Significant difference with DA scores on Moz Chrome app VS Website Tool
I saw a big difference between DA scores on Moz Chrome app VS Website Tool. Which DA score is the correct one? I personally believe the scores on Moz's site is most accurate. Do you happen to have issues syncing the scored to your Chrome app? Chrome App VS. Moz Research Tool (https://moz.com/researchtools/ose/)
Moz Bar | | iPrice_Marketing
Moz.com: 92 / 88 Here's a screenshot of the difference: https://ipricegroup-my.sharepoint.com/:i:/p/jeremy_chew/ESy9lzUTC3lOl8o93Sx6LsUB8mYXo8LmYuUj2aa0xLXi1A?e=A6OrXs This was evident in other websites too:
Priceza (priceza.com.my😞 38 / 24
Shopback (shopback.my😞 43 / 41
Cuponation (https://www.cuponation.com.my/😞 27 / 250 -
Community Discussion - What's Been Your Experience With Moz Content?
When the content developed Moz Content, I was excited as can be about having another tool in the content marketing and content strategy repertoire. I knew it could and would help marketers better identify the content they should be creating and make it easier for them to move the needle for their brands. Since it's been available, I've had fun using Moz Content, seeing it as a great vehicle for flattening the learning curve for content ideation and creation. In a recent post, Here's How I'm Using Moz Content for Mining Local Link Opportunities, David Farkas described how brands can use Moz Content to better create localized content. I'd like to know how you're using it, or if you're using it: Have you tried Moz Content? And if not, what's stopping you? If you have used it, what are you really liking? What would you change? What, if any, additional features you'd like to see added? What tips can you share for helping others get the most out of the tool? Looking forward to reading the comments below.
Moz Bar | | ronell-smith3 -
Ask moz staff
Was trying to crawl a page and the moz on page grader tells that URL is not accessible. The URL is accessible thorugh my mobile and desktop. Any idea what is the issue ? The URL is https://www.practo.com/singapore/dentist IGrzzVe
Moz Bar | | ozil0 -
Moz Report-Canonical
Moz crawl reported in issues: Rel Canonical: using rel=canonical suggests to search engines which url should be seen as canonical. But we have sorted out this a long time ago. Is there a way where can I verify if the moz crawl is wrong or if we really do not specify the canonical?
Moz Bar | | Rebeca10 -
Is there a way to get Page Authority values included in the Crawl Diagnostic .csv export?
Would be nice to have these values included so that you can sort by Page Authority. 4uF6efx.png
Moz Bar | | WebReputationBuilders0 -
Moz crawl sees meta description but there are none
I have a new site I have run a starter crawl on. The crawl came back saying some of the pages do indeed have meta descriptions. When I go to the same page and use the Moz Chrome toolbar it says they do not have meta descriptions. I also know they do not have meta descriptions. Are there any instances in which the Moz crawler would see them when they are not there?
Moz Bar | | SBXMedia0