Googlebot encountered extremely large numbers of links on your site??? How Do I resolve this?

[GlobalFlex]

I am working on a site with over 30 million pages. Every time I get about  One Million indexed I get a Message in the Google Webmasters Tools saying "Googlebot encountered extremely large numbers of links on your site"

The indexing then starts dropping like a Rock. I need to get the site indexed. Please Help!

[AlanBleiweiss]

Kenneth

I work with extremely large sites quite often.  There's no single answer to this because it depends on what's going on as to why the Googlebot is breaking down in its crawl.  For example - how many links exist on any single page?  Is it 100, 300, 1000 or more?  The more links on every page the more likely the bot will choke, though it's a lot better than it used to be.

Does the site validate for markup?  Or could there be choke-points due to validation errors?

Is the content organized in an intelligent funnel structure, or is everything one level off the root domain?

Is there only one way for the bot to navigate deep into the site, or are there multiple methods to get down deep?

Is some of the content only linked from within in a way that many of those links are not discovered until the bot has to first go through six or eight other layers, some of which could be timing out just when the bot gets there?

How many quality inbound links point to pages deep within the site?

These are all questions that need to be asked and answered and that's just scratching the surface of the problem potential.

The most important thing is to try and think like the bot - if I go here, will I become overwhelmed?  if I go here, will I hit a road-block?