Why Is SEOMOZ No Longer crawling All Of My Site
-
Hi all,
I joined Seomoz over a month ago and Roger has been crawling all of the pages on the site approx 20 pages. Through out the last few weeks I have been working on the errors and notices identified by Roger. However, this week Roger has only re-crawled 1 page and is not picking up all the other pages.
Has any one come across this problem. can you recommend any thing to resolve it?
Many thanks in advance....
-
Hi Greg,
I haven't heard of that one before..hate to sound like a broken record, but best to email the help desk for this one too. Thanks!
-
I am not sure If I have the same problem... I previously had 46 pages crawled and added a couple in the last week. So the last crawl should have said 48. Oddly though, it reads N/A.
I would think a little tooltip should explain what N/A means there. Does anyone know?
-
Hi Echolls, so sorry to hear that. Can you send an email to help at seomoz dot org and let our help desk know? Thanks!
Keri
-
Just chiming in to add that I have the same problem going right now. Worked totally fine for a few days with 200+ pages crawled, now just down to one.
-
The re-crawl today has now picked up all the pages on the site. The problem was only for a few days.... Many thanks
Danny
-
There's a known bug at the moment that we're investigating that has some crawls suddenly dropping down to only a page or two. I believe both of you have sent your information to the help desk, and they'll help you from there. I'm making this comment so anyone else searching the forums in the next couple of days can see this answer as well and just contact the help desk directly.
-
Hi Seb,
Yeah, sent it through now waiting for a reply. Let me know if you have any joy and if it's an error on their behalf?
Many thanks
Danny
-
I did come across that answer but none of them really applies to me. Have sent a help request. Many thanks for your help.
Danny
-
Hi,
I am having the same Problem: Crawled Pages droped from 10k to 1 Pages... send a request for help to help@ allready...
Seb.
-
Here are some common reasons why only one page of your site may be crawled http://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-only-crawled-one-page. For more specific help, sending an email to the help team at help@seomoz.org is the best way to get your particular issue looked at.
Thanks!
Keri
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
WEbsite cannot be crawled
I have received the following message from MOZ on a few of our websites now Our crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster. I have spoken with our webmaster and they have advised the below: The Robots.txt file is definitely there on all pages and Google is able to crawl for these files. Moz however is having some difficulty with finding the files when there is a particular redirect in place. For example, the page currently redirects from threecounties.co.uk/ to https://www.threecounties.co.uk/ and when this happens, the Moz crawler cannot find the robots.txt on the first URL and this generates the reports you have been receiving. From what I understand, this is a flaw with the Moz software and not something that we could fix form our end. _Going forward, something we could do is remove these rewrite rules to www., but these are useful redirects and removing them would likely have SEO implications. _ Has anyone else had this issue and is there anything we can do to rectify, or should we leave as is?
Moz Pro | | threecounties0 -
Site Crawl 4xx Errors?
Hello! When I check our website's critical crawler issues with Moz Site Crawler, I'm seeing over 1000 pages with a 4xx error. All of the pages that are showing to have a 4xx error appear to be the brand and product pages we have on our website, but with /URL at the end of each permalink. For example, we have a page on our site for a brand called Davinci. The URL is https://kannakart.com/davinci/. In the site crawler, I'm seeing the 4xx for this URL: https://kannakart.com/davinci/URL. Could this be a plugin on our site that is generating these URLs? If they're going to be an issue, I'd like to remove them. However, I'm not sure exactly where to begin. Thanks in advance for the help, -Andrew
Moz Pro | | mostcg0 -
Functionality of SEOmoz crawl page reports
I am trying to find a way to ask SEOmoz staff to answer this question because I think it is a functionality question so I checked SEOmoz pro resources. I also have had no responses in the Forum too it either. So here it is again. Thanks much for your consideration! Is it possible to configure the SEOMoz Rogerbot error-finding bot (that make the crawl diagnostic reports) to obey the instructions in the individual page headers and http://client.com/robots.txt file? For example, there is a page at http://truthbook.com/quotes/index.cfm month=5&day=14&year=2007 that has – in the header -
Moz Pro | | jimmyzig
<meta name="robots" content="noindex"> </meta name="robots" content="noindex"> This page is themed Quote of the Day page and is duplicated twice intentionally at http://truthbook.com/quotes/index.cfm?month=5&day=14&year=2004 and also at http://truthbook.com/quotes/index.cfm?month=5&day=14&year=2010 but they all have <meta name="robots" content="noindex"> in them. So Google should not see them as duplicates right. Google does not in Webmaster Tools.</meta name="robots" content="noindex"> So it should not be counted 3 times? But it seems to be? How do we gen a report of the actual pages shown in the report as dups so we can check? We do not believe Google sees it as a duplicate page but Roger appears too. Similarly, one can use http://truthbook.com/contemplative_prayer/ , here also the http://truthbook.com/robots.txt tells Google to stay clear. Yet we are showing thousands of dup. page content errors when Google Webmaster tools as shown only a few hundred configured as described. Anyone? Jim0 -
Crawl Diagnostics - Crawling way more pages than my site has?
Hello all, I'm fairly new here, more of a paid search guy dabbling in SEO on the side. I have a client that I have in SEOMoz and the Crawl Diagnostics report is showing 10,000+ pages crawled and I think the site has at most 800 pages (e-commerce site using freewebstore.org as the platform). Any reasons this would be happening?
Moz Pro | | LodestoneGen0 -
Getting your site totally indexed by SEOMOZ
Hi guys! Ijust started using SEOMOZ software and wondered how it could be that my site has over 10.000 pages but in the Pro Dashboard it only indexed about 1500 of them. I've been waiting a few weeks now but the number has been stable ever since. Is there a way to get the whole site indexed by SEOMoz software? Thanks for your answers!
Moz Pro | | ssiebn70 -
SEOmoz Toolbar - Am I the only one who figured this out?
I have been using the SEOmoz toolbar for a while. In the course of a day, and with so many projects going on I end up having about 15-20 tabs in Firefox open at once. Because of the toolbar, my browsing session slows to a crawl and I hate to disable the toolbar because then I have to shut down the 10 projects I am working on. I usually close down my browser when I can't take it anymore and it feels like I am browsing on my 1990's palm device. What I decided to do is install the toolbar on Google Chrome. I don't use this browser very often so now whenever I need to check out a website, I just use Chrome instead of Firefox. Maybe there is a way to turn off the toolbar that I don't know about. Any other toolbar tips?
Moz Pro | | Czubmeister0 -
I have signed up for an seomoz trial .......
i have signed up for an seomoz trial but tried to access my account from a different laptop yesterday and could not a login. Is the trial version only accessible via the laptop i set it up from?!
Moz Pro | | GreatGifts4Kids0 -
Seomoz staff help needed
I have this new client new domain new website, www.ard.uk.com when seo moz crawls the site it says it has 24 million links mozrank of over 7, looks amazing but impossible can someone as seomoz have a look please
Moz Pro | | francesco-2850160