Why are only two pages of my site being crawled by MOZ?
-
MOZ is only crawling two pages of my site even though there are links on the homepage, and the robot.txt file allows for crawling. Our site address is www.hrparts.com.
-
I had to remove subdomains from being crawled on the site, and the issue has been resolved. The 301 redirect was an issue, causing MOZ to stop crawling at only two pages, the http and https pages.
-
Thank you for your response, Lauren. Our site only has one major feature that is in Javascript, but the rest of the site is not. We do have a 301 redirect from hrparts.com to www.hrparts.com which is where I believe the issue might be coming from. I am curious whether MOZ has an issue with a 301 redirect in a campaign?
-
Hi there!
Thanks for writing in and sorry about the trouble!
I took a look at your Site Crawl results and your site, and I believe this is a result of our crawler's inability to work with Javascript. Any client side loaded content will not be rendered to our crawler as we are only seeing what is rendered by your server.
Our crawler works by parsing the source code of your site, looking at HTML elements. If your site is primarily Javascript, then the data you get back with regards to the crawl report won't be completely accurate because of this. There's no real workaround that I can recommend for this one, since it is a technical limitation of our tools, but there are some good blog posts and discussions in the Q&A about this if you head over to our Help Hub.
While the tools and data that rely on our crawl of your site may not return the best results because of that Javascript, your keyword rankings and link profile should work just fine.
You might also want to check out a few tools that are compatible with Javascript, like Botify or Screaming Frog.
https://www.onely.com/tools/wwjd/ is also a handy tool that makes it super easy to see where links are added or removed by javascript.
I'm really sorry I can't be of more help here; Please let me know if there's anything else I can help with, or if there are any follow-up questions you might have!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Solved Site Crawl Won't Complete
How can I start/restart a new site crawl? I requested one 2 days ago on one of my sites, and it won't complete. It's only 150 pages -
Product Support | | PaulBarrs0 -
Remove account from MOZ
I have subscribed for 1 month free Moz trial account. It was free but to subscribe this account I need to put my credit card details. Suddenly at the end of this 1 month term period, Moz tried to cut 149 Dollar from my account without notifying me. But they could not because there were insufficient funds on my credit card. I haven't used any services and would like to remove my account from Moz service? Would appreciate your assistance.
Product Support | | edgarsb0 -
I have subscribed for 1 month free Moz trial account. i need to delete my account permanently
i can not delete my account from account setting , so what should i do??
Product Support | | woline20080 -
Moz Campaign Not Updates
For some reason my Campaign hasn't run this week. It was supposed to run on Sunday, and it is now Tuesday. Just wondering if someone from support can take a look. Thanks.
Product Support | | TheWebMastercom1 -
Moz Pro Reports - Can I create Monthly client reports?
I am just trying to work out if I am able to create monthly SEO reports? I see that when setting up a custom report in my campaigns, i can get it to run daily, weekly or monthly. But i'm just double checking that if i set the reports to run monthly, its going to collect all of the data from that full month, not just grab the data from the most recent week? Thanks
Product Support | | SWD.Advertising0 -
How can I get the August SEO Moz receipt?
I need to get a copy of the seo moz receipt for August 2013. I did not receive it. Please advise. Thanks!
Product Support | | tucweb0 -
How to force MOZ crawlers to search in the right area for local SEO
We do local SEO over here and so our keywords are always followed by a city name, like so: "auto repair+Springfield". If there's ever any confusion within SERPS which "Springfield" the searcher is looking for we can depend on Google to pull query location to assume it's the nearest to them (it does, and is usually correct). And furthermore, if the office ever needs to check for rankings by hand (we're not in Springfield), we just set the query location to "Springfield" and our results match their. However, MOZ doesn't set a query location. Which makes sense, I suppose, because their crawlers are up in the Bay. What I'm saying is that oftentimes there's a discrepancy between how MOZ tells me they rank, and how I see them ranking, the business owner sees them ranking, and how the customer finds them ranking. As such, we can't send them reports through MOZ 😞 Has anyone else had this problem and found a solution? Should the keyword be followed by a zip code instead of a city name? This seems like it would solve the location query problem but it may not provide the same results as service+city name.
Product Support | | cwtaylor0 -
Question about Moz Point
Today I updated my profile and now it is 100% complete. But still my Moz Point is showing as 0(Zero). Please let me know when it'll get updated?
Product Support | | lakhyajyoti0