Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Moz crawler only crawls one page?!
-
Hello there,
I'm using Moz for a while and I'm very pleased with the tool and community. But for the first time I encountered a problem. We are trying to run a crawler for a client's website but only one page (only the homepage) was crawled.
We tried to do a test on a more detailed level (maybe there is something wrong with the homepage). My campaign test's crawl came back for the Producten folder (level deeper than homepage), and it was also only a 1 page crawl with a 200 status. I did look at the robots.txt file now, and it is very restrictive, but there is nothing that I can clearly see that would explain why the crawl isn't working.
Hopefully someone can point us at the right direction.
Thanks in advance,
Jeremy
-
Hey Jeremy,
I took a look at your campaign and it looks like Lynn's suggestion that Javascript may be blocking the crawl is correct. Unfortunately, our crawler isn't sophisticated enough to parse Javascript and all of the links from the homepage are hidden within the Javascript navigation so we aren't able to find them to crawl past the homepage. I'm afraid it looks like our crawler isn't compatible with this particular site.
I apologize for the inconvenience this causes! Please let me know if I can help you with anything further.
-
Thanks!!! I will take a look at the dev blog post
-
Hi,
It can happen for a number of reasons, in my experience usually due to either unintentional blocking of the page from crawling or from being a JavaScript heavy/only site which doesn't allow content to be easily read by the moz crawler. Check out this dev blog post for a rundown on most issues and solutions. Hope it helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz not updating the spam score metrics
Hi Experts, I have done everything to come over the following metrics flagged as spam on my website by Moz, few months ago. But, Moz has not updated the spam score yet. ✓Low MozTrust or MozRank Score - improved from 2 to 4.5
Moz Bar | | jamesh.rich01
✓Large Site with Few Links - My website have more than 6K backlinks
✓Small Proportion of Branded Links - My website have a good amount of branded backlinks
✓Thin Content - Every webpage on website has more than 500 words content
✓External Links in Navigation - There is no external link in navigation other than social media links
✓No Contact Info - The proper address has alreay been placed on website footer
✓Low Number of Pages Found - I am wondering if there are any standard score or number of links to reach to remove these flag?
Also, please suggest some ways that will help me improve moz spam score at faster rate. Thanks for your help in advance!"0 -
Need to solve "Oops our crawlers were unable to access" url for new campaign
I'm putting the url designfirstkitchenandbath.com and getting the "oops! our crawlers were unable to access the site. Since this site is a potential client, which shows up online, I can't get access to fix the code, plus while I can write a little html I don't feel comfortable working with hard, live code on someonelse's site. Anyone have a simple solution?
Moz Bar | | alisacromer0 -
Error 406 with crawler test
hi to all. I have a big problem with the crawler of moz on this website: www.edilflagiello.it. On july with the old version i have no problem and the crawler give me a csv report with all the url but after we changed the new magento theme and restyled the old version, each time i use the crawler, i receive a csv file with this error: "error 406" Can you help me to understan wich is the problem? I already have disabled .htacces and robots.txt but nothing. Website is working well, i have used also screaming frog as well.
Moz Bar | | ArchieDonnithorne1 -
Moz Bar Not Showing DA?
Hello all, This could be something to do with our site or the Moz bar on Chrome, I just need to know which it is so if it is our website we can look into it further. On certain sections of our website the Moz bar doesn't display any Domain Authority, not even zero, the bar just isn't present. These types of pages are php which pull in data through a feed daily. Speaking to an SEO expert they said it could be where the page is being updated so frequently, or it could be something more sinister and technically not quite right. Does anyone have any ideas? Is the Moz bar just not working for these types os pages or is it more likely something to do with my site? Ironically it's these pages which I'm having trouble with that are not showing in SERPs! Thanks! 3Foorka
Moz Bar | | HB177 -
Duplicate Page and Title Issues
On the last crawl, we received errors for duplicate page titles and some duplicate content pages. Here is the issue: We went through our page titles that were marked as duplicate and changed them to make sure their titles were different. However, we just received a new crawl this week and it is saying there are even more duplicate page title errors detected than before. We're wondering if this is a problem with just us or if it has been happening to other Moz users. As for the duplicate content pages, what is the best way to approach this and see what content is being looked at as a "duplicate" set?
Moz Bar | | Essential-Pest0 -
Checking Do-follow using Moz Bar
Does anyone know how to check if a link is a do-follow or no-follow using the Moz bar? I believe there's a function that highlights and color codes the link so as to tell? Can't seem to turn this function on in my Moz Bar settings. Am I missing something completely obvious?
Moz Bar | | Gavo0 -
How do I export my keywords from Moz?
Simple question: once you've built up a big set of keywords within Moz, how do you export it back out to use in other places?
Moz Bar | | tcolling0
