Why are only two pages of my site being crawled by MOZ?
-
MOZ is only crawling two pages of my site even though there are links on the homepage, and the robot.txt file allows for crawling. Our site address is www.hrparts.com.
-
I had to remove subdomains from being crawled on the site, and the issue has been resolved. The 301 redirect was an issue, causing MOZ to stop crawling at only two pages, the http and https pages.
-
Thank you for your response, Lauren. Our site only has one major feature that is in Javascript, but the rest of the site is not. We do have a 301 redirect from hrparts.com to www.hrparts.com which is where I believe the issue might be coming from. I am curious whether MOZ has an issue with a 301 redirect in a campaign?
-
Hi there!
Thanks for writing in and sorry about the trouble!
I took a look at your Site Crawl results and your site, and I believe this is a result of our crawler's inability to work with Javascript. Any client side loaded content will not be rendered to our crawler as we are only seeing what is rendered by your server.
Our crawler works by parsing the source code of your site, looking at HTML elements. If your site is primarily Javascript, then the data you get back with regards to the crawl report won't be completely accurate because of this. There's no real workaround that I can recommend for this one, since it is a technical limitation of our tools, but there are some good blog posts and discussions in the Q&A about this if you head over to our Help Hub.
While the tools and data that rely on our crawl of your site may not return the best results because of that Javascript, your keyword rankings and link profile should work just fine.
You might also want to check out a few tools that are compatible with Javascript, like Botify or Screaming Frog.
https://www.onely.com/tools/wwjd/ is also a handy tool that makes it super easy to see where links are added or removed by javascript.
I'm really sorry I can't be of more help here; Please let me know if there's anything else I can help with, or if there are any follow-up questions you might have!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Solved Why is MOZ crawl taking so long?
I began my site crawl on November 3rd and now it is November 7th and it is still "in progress". Why is this happening?
Product Support | | CarisaS_Wenda0 -
Site DA and PA is not increasing
Hy, My site best water softener DA is Still 1. I am Publishing the content regularly, Site age is 6 months But Still DA and PA are 1. Why its not increasing. Is there any way to increase DA and PA Of my site?
Product Support | | ikzayahot0 -
Our client changed their domain name. Can we replace without losing the data for the old domain in MOZ ? We are at our campaign limit would the old domain count as a campaign?
Our client changed their domain name. Can we replace it without the new domain without losing the data for the old domain in MOZ ? We are at our campaign limit would the old domain count as a campaign?
Product Support | | ThomasCenterInc1 -
Analytic goals in Moz
We should have the ability to see any and all conversions goals, as defined in analytics, in Moz. After all, the whole point of marketing, SEO, PPC, Social, etc. is to drive sales or leads that create sales right? So, why should we have a dashboard tool that doesn't show us the 1 thing that ultimately matters the most? I understand that Moz simply wasn't designed to do this, but I think that should change. After all, we are in this to make money I hope. My 2 Cents. Micha
Product Support | | Multiverse-Media-Group1 -
Could not connect to analytics.moz.com
Hi All, For 2 days I cannot access my moz analytics. I can log in, but when I click the Moz Analytics button, which redirects me to: http://analytics.moz.com/pro/home I receive message that chrome could not connect. Screenchot attached. Rgds! Screen-shot-2014-05-24-at-11.40.51-AM.png
Product Support | | mefendulov1 -
I'm not canceling my MOZ subscription, but just so I know: if I want to cancel my MOZ subscription, can I do this per month, or per year?
If I want to cancel after six months, do I have to pay for the rest of the year or can I cancel per month?
Product Support | | Sophie-Kool0 -
How do you delete a campaign in Moz Analytics ? (i set up 2 campaigns for the same project by accident)
Hi How do you delete a campaign in Moz Analytics ? (i set up 2 campaigns for the same project by accident) cheers dan
Product Support | | Dan-Lawrence1 -
503 error on Moz
Is any one else getting a 503 error (I'll copy it below) whilst trying to get on Moz, I can get to the home page but going to internal links I get the error, I managed to get onto Q&A via an external link. I've been having some troubles the last few weeks with logging on etc. Most importantly Is Roger ok ? Error 503 Service Unavailable Service Unavailable Guru Meditation: XID: 1257248781 Varnish cache server
Product Support | | GPainter0