Crawl slow again
-
Once again the weekly crawl on my site is very slow. I have around 441 pages in the crawl and this has been running for over 12 hours. This last happened two weeks ago (ran for over 48 hours). Last week's crawl was much quicker (not sure exactly how long but guessing an hour or so).
Is this a known issue and is there anything that can be done to unblock it? Weekends are the best time for me to assess and respond to changes I have made to my site so having this (small) crawl take most of the weekend is really quite problematic.
Thanks.
Mark
-
Your best bet right now is to contact the SEOmoz help desk with an email to help@seomoz.org. They may (not sure) be able to have your crawl start on a different day of the week so it's ready for you by the weekend, and should be able to answer all of your crawl-related questions in general.
-
Thank you for the response. I do not have a robots.txt file. I'm not following how the Google crawl rate effects this (but would love to learn why). I thought that is only used to define how often Google itself crawls my site and has nothing directly to do with SEOMoz. Also, it is not how often the SEOMoz crawl takes (it starts on time each Saturday) but, rather, how long it takes to complete. For two of the last three weeks it has taken almost 48 hours from start to finish.
Thanks again.
Mark
-
Have you tried changing the crawl rate in Google Webmaster Tools? Also make sure your robots.txt isn't specifying a crawl rate delay
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Who wants to help go over my crawl diagnostics via skype?
I have run a crawl diagnostic on my site and have 194 errors and most of them are 404 errors in wordpress. Not sure why, but many of my pages had name changes (possibly a permalinks issue) but I have no idea how to fix it. I had 5 duplicate page titles, and 1 tile missing or empty. 72 crawl notices found (2 permanent redirect, 17 blocked by robots, 53 rel canonical) 19 Crawl warnings were found Who wants to have some fun?
Moz Pro | | starkSEO0 -
Crawl credits how to buy more?
Just wondering if there is a way of increasing, my 2 crawl credits per day limit?
Moz Pro | | aussieseoguy0 -
Crawl Diagnostic | Starter Crawl taken 14hrs.. so far
We started a starter crawl 14hrs ago and it's still going, can anyone help on why this is taking so long, when it says '2 hrs' on the interface.. Thanks, Rory
Moz Pro | | RoryMacDonald0 -
Settings to crawl entire site
Not sure what happened but I started a third campaign yesterday and only 1 pages was crawled, The other two campaigns has 472 and 10K respectively. What is the proper setting to choose in the beginning of campaign setup to have the entire site crawled. Not sure what I did different and I must be reading the instructions incorrectly. Thanks, Don
Moz Pro | | NicheGuy210 -
Links & page authority crawl
I see the links and page authority have not been updated in over a month... does anyone know how often it gets updated?
Moz Pro | | nazmiyal0 -
Why won't scheduled crawl of my site begin?
I currently have a campaign running on SEOMoz for over a month. It has been showing that a crawl was scheduled to start on 12/21. Now it's 12/23 and there has not been a new crawl, and it still says scheduled for 12/21.. Anyone know why this is happening or how to fix it? Thanks
Moz Pro | | Prime850 -
Can I change the crawl day ?
Hi All I hope there is a simple solution to this - we have a number of campaigns setup which are all crawled, and therefore updated, on different days of the week. We review these weekly and it would be much easier if they were all crawled on the same day. Is it possible to change the crawl day for some campaigns? Thanks Roy
Moz Pro | | bluelogic0 -
SEOMoz's Crawl Diagnostics showing an error where the Title is missing on our Sitemap.xml file?
Hi Everyone, I'm working on our website Sky Candle and I've been running it as a campaign in SEOmoz. I've corrected a few errors we had with the site previously, but today it's recrawled and found a new error which is a missing Title tag on the sitemap.xml file. Is this a little glitch in the SEOmoz system? Or do I need to add a page title and meta description to my XML file. http://www.skycandle.co.uk/sitemap.xml Any help would be greatly appreciated. I didn't think I'd need to add this. Kind Regards Lewis
Moz Pro | | LewisSellers0