Get Google To Crawl More Pages Faster on my Site

[Intergen]

We opened our database of about 10 million businesses to be crawled by Google.

Since Wednesday, Google has crawled and indexed about 2,000 pages.  Google is crawling us at about 1,000 pages a day now. We need to substantially increase this amount.

Is it possible to get Google to crawl our sites at a quicker rate?

[NakulGoyal]

Mark If you have had your category level or city/state level pages, whatever your architecture is, you'd see your detail pages getting crawled pretty quickly. Give it a week. I would also work on getting some deep links to your website. Google has mentioned several times, PR is a direct measure of how deep and how often Google crawls your website. So I would suggest working on building the domain authority and brand in terms of backlinks to the website. Other then that, as bstone81 said, generate sitemaps and sitemap of sitemaps and submit to Google.

[tdawson09]

Two things you can do:

1. Submit a site map with all those pages. They suggest breaking the sitemaps into 50,000 page groups.

2. Use the "Fetch as Google" feature in GWT and fetch your root domain then click the box to submit the root domain along with all the linked pages. This will submit the request to Google to be indexed.

[TommyTan]

Hi Mark,

Google only spends a certain amount of time to crawl and there is no way to tell them to crawl more or faster.  If you have a lot of pages, it might take longer.

One way to make them crawl faster may be optimize your internal linking.  If you have links pointing to other pages in your website, it makes it easier to crawl.

Other than work arounds, there are no ways to tell Google to crawl faster or more pages.  They have their own speed.

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=34439