Why are only two pages of my site being crawled by MOZ?
-
MOZ is only crawling two pages of my site even though there are links on the homepage, and the robot.txt file allows for crawling. Our site address is www.hrparts.com.
-
I had to remove subdomains from being crawled on the site, and the issue has been resolved. The 301 redirect was an issue, causing MOZ to stop crawling at only two pages, the http and https pages.
-
Thank you for your response, Lauren. Our site only has one major feature that is in Javascript, but the rest of the site is not. We do have a 301 redirect from hrparts.com to www.hrparts.com which is where I believe the issue might be coming from. I am curious whether MOZ has an issue with a 301 redirect in a campaign?
-
Hi there!
Thanks for writing in and sorry about the trouble!
I took a look at your Site Crawl results and your site, and I believe this is a result of our crawler's inability to work with Javascript. Any client side loaded content will not be rendered to our crawler as we are only seeing what is rendered by your server.
Our crawler works by parsing the source code of your site, looking at HTML elements. If your site is primarily Javascript, then the data you get back with regards to the crawl report won't be completely accurate because of this. There's no real workaround that I can recommend for this one, since it is a technical limitation of our tools, but there are some good blog posts and discussions in the Q&A about this if you head over to our Help Hub.
While the tools and data that rely on our crawl of your site may not return the best results because of that Javascript, your keyword rankings and link profile should work just fine.
You might also want to check out a few tools that are compatible with Javascript, like Botify or Screaming Frog.
https://www.onely.com/tools/wwjd/ is also a handy tool that makes it super easy to see where links are added or removed by javascript.
I'm really sorry I can't be of more help here; Please let me know if there's anything else I can help with, or if there are any follow-up questions you might have!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Why are so many subdomains detected in moz? even though I never made it.
I saw it in the Links -> Top Pages menu, here many sub domains were detected. even though I never made it, is that normal? multiple subdomains detected:
Product Support | | sem001234
w4bd5da.dummy.com
09x.dummy.com
7h5a.dummy.com
8d1y8ec.dummy.com
sfxpwzx.dummy.com
px0ke.dummy.com
uaj.dummy.com0 -
Refund From MOZ for Cancelled Account-How To Get Refund, Phone Number for MOZ Corporate Office?
I have written MOZ several times requesting account cancellation procedures but they don't seem interested in responding. I pre-paid my account in October 2019, paying for one year until October 2020. I would like to cancel my account and get a refund of the pro-rated unused amount. After not getting a response, I searched for a corporate phone number for MOZ. No phone number can be found. I am somewhat concerned as legitimate companies have phone numbers. Can someone advise me as to how to get my refund? Also, I subscribe to MOZ local. I need to know of canceling MOZ pro will inadvertently cancel MOZ local. Thanks,
Product Support | | Kingalan1
Alan0 -
Website can't be crawled
Hi there, One of our website can't be crawled. We did get the error emails from you (Moz) but we can't find the solution. Can you please help me? Thanks, Tamara
Product Support | | Yenlo0 -
Can 2 Moz profiles be merged?
I'm very excited to have received my promotion into the Moz Specialist category (yea me!) , but I've discovered that I have another Moz profile dating back to 2011. Can they be merged together?
Product Support | | julie-getonthemap0 -
Moz has stopped crawling my site
Hello, We have been using Moz Pro for over a year on our site and in the last month noticed that our site is not being crawled anymore. I took a look in Google Search Console, and everything seems to be fine there, so I think it is just the Moz tool that is not working. Has anyone else experienced this? Are there any tips for troubleshooting it? Thank you, Adam
Product Support | | cwells0 -
Creating Multiple Moz Accounts
Hi there, due to my work - and for customer service - I need to be able to set up multiple Moz accounts for each client, providing their own contact/CC info. Can you tell me if Moz allows this (I keep getting the message that I'm registered and can upgrade), or do I have to login from a different computer each time? Ultimately, the client will be administering their own account, but I want to set them up to get them started. Thanks, Alison alison@snap360.ca
Product Support | | kdickson0 -
Why is Moz report showing duplicate content?
Dear Moz Community Our weekly Moz crawl diagnostic repoart is showing a significant increase in "Duplicate Page Content" errors for article pages that have unique content, unique file names, unique META title/descriptions, and unique H1 tags. Where could the duplication be coming from? Thanks for your help.
Product Support | | BoomDialogue690
