How Often is Site Crawled

[VT_Pete]

Good morning-

I saw some errors in my first crawl and immediately removed the pages from my website.  I then re-created my XML sitemap and uploaded to Google.

The question I have is will the site be crawled to recognize the changes in the next day or so?  The pages were just placed on the site as test pages and never removed.  The initial crawl that notified me it was done found the errors and were removed.

Thanks for your help.

Peter

[Matt-Williamson]

Oops sorry I misunderstood - thought you meant Google, but at least Keri jumped in there and gave you the answer

[KeriMorgret]

SEOmoz will crawl the site once a week. You can go to research tools and select a crawl on demand of up to 3000 URLs from a domain if you'd like a crawl sooner.

[VT_Pete]

Thanks Matt.  What I meant to ask was how often will SEO Moz crawl the site?

[Matt-Williamson]

There is no set time limit to how often a site is crawled by Google - different sites have different crawl rates depending on factors such as how popular they are. For instance a well ranked, popular news site with content that is regularly being updated and added will have a crawl rate that is higher than a small personal site.

Have a look at this article in regards to ways you can encourage Google to crawl your site more often - http://www.searchenginejournal.com/10-ways-to-increase-your-site-crawl-rate/7159/ (an old article but still applies)

Have you thought about using the URL removal tool in Google Webmaster Tools to quickly remove the test URLs -  http://support.google.com/webmasters/bin/answer.py?hl=en&answer=164734

And finally in relation to speeding up the crawling of specific pages on your site you can use the fetch as Googlebot tool > enter a URL > click submit URL to index to speed the crawling of that content up. Here is a great article on this much underused tool - http://searchenginewatch.com/article/2198245/How-to-Force-Google-to-Recrawl-Your-Site