About Google Spider
-
Hello, people!
I have some questions regarding on Google spider.
Many people are saying that "Google spiders only have US IP address."
Is this really true?
But I also saw video from Google's offical blog and it said "Google spider come from all around the world."
At this point I am really confused.
Q1) I researched and it seems like Google spiders have only US IP address. THen what does exactly mean by "Google spider come from all around the world."?
Q2) If Google spider have only US IP address, what happen to site which use IP delivery?
Is this means that Google spider always redirect to us site since they only have US IP?
Can anyone help me to understand??
One more questions! When Google analyzing for cloaking issue, do you think Google analyze when spider crawls the site or after they crawled the site?
-
I think some of the confusion may be due to Google's primarily using IP addresses assigned to their headquarters in Mountain View, California. Google has many (around 20) data centers located outside the US. I recall reading an article whereby at times they used their Mountain View IPs from centers around the world. For security reasons they do not wish the location of all their data centers to be known.
I researched this topic before and I was unable to locate any official information from Google. It would only seem reasonable they crawl from all over the world. If they didn't, then a lot of sites which use geo-based targeting for site navigation would not have most of their content indexed. While it's true a sitemap could be used to overcome the issue, many sites do not use sitemaps and they still get indexed.
-
I do not believe this is true, Google has data centers all over the world in which they crawl from.
Google does not only have spiders crawling from US data centers.
I also have the feeling crawls are based on many factors such as link diversity per region, TLD of domain per region, PR user (still a crawling factor imo) and many more factors.
Overall do not stress Google can crawl from various regions all over the globe, I would be more worried about geo server location and TLD of your domain and also local links.
Kind Regards,
James Norquay.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Incorrect Youtube metadescription in Google SERP
One of our client's Youtube video is showing a competitor's meta data on the Google search results page? It looks like it is pulling from videos in the right-hand rail of the Youtube page. Is there any way this can be controlled/changed? If so, how? The client is Deep South Crane. When you perform a search for "Deep South Heavy Hitters", the correct video appears in the search results. However, the meta-description is pulling from a competitor's Youtube. Any insight as to why this is happening and how I can change it would be greatly appreciated. Thank you.
Technical SEO | | JaredBroussard0 -
How long does it take for Google for deindexing pages?
Hi mozzers, We just launched a mobile website(parallel) and realized that it created many duplicate content with desktop URLs. I decided to add name="robots" content="No index, No follow" /> to the entire mobile site. My only concern is that I am still seeing the mobile site indexed when it's been almost a week I added these tags. Does anyone know how long it takes google to deindex your content? Thanks
Technical SEO | | Ideas-Money-Art0 -
Influencing Google Instant Preview
Hello there! I have been looking at how our articles are shown in Google Instant Previews. While the description in the SERPs picks up the first paragraph from the article from the main table/div body, the Instant Preview highlights a text ad in the top right of the side column. According to Google https://sites.google.com/site/webmasterhelpforum/en/faq-instant-previews Q: How can I influence the text highlighted in the preview image?
Technical SEO | | CleverPhD
A: The highlighted text is automatically chosen based on the user's search query. Only text that is visible on the page can be selected for highlighting. Google is highlighting the same block of text from an ad space over and over again. "Download our Free Guide ...." vs the first paragraph in the article. Any ideas folks?0 -
Google Index Speed Opinions
Hello Everyone, Under normal circumstances, new posts to my site are indexed almost instantly by Google. I know this because an occasional search with quotation marks surrounding the 1st paragraph of text displays my newly published page. I use this tactic from time to time to ensure contributors aren't syndicating content. My question is this: I've noticed over the last day or so that my newly published articles are not yet indexed. For example, an article that was published over 24 hours ago does not appear to be indexed yet. Is this cause for concern? Is there an average wait time for indexation? XML issue? Thanks in advance for the help/insight.
Technical SEO | | JSOC0 -
How to tell google about the changes
Hello All this is my first question here. we have recently changed our website design and backend. so the problem now is in google it is showing the old url's and when ever a client come from google he is prasented with a 404 error page if this happenes many times. will google penalise our site ? if yes. how to prevent this from happen. waiting for some replies.. thanks
Technical SEO | | idreams1 -
Quality analytics without Google?
I'm trying to find a program or a site that will give me quality traffic data. If I'm going into a presentation or simply a round one meeting I like to prepare a 1 pager with info I've found. Without authentication access to their Google Analytics I feel like I'm a step behind. Any Help would be greatly appreciated. -JoeGrrrcia
Technical SEO | | JOEGRRRCIA0 -
Google indexing page with description
Hello, We rank fairly high for a lot of terms but Google is not indexing our descriptions properly. An example is with "arnold schwarzenegger net worth". http://www.google.ca/search?q=arnold+schwarzenegger+net+worth&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a When we add content, we throw up a placeholder page first. The content gets added with no body content and the page only contains the net worth amount of the celebrity. We then go back through and re-add the descriptions and profile bio shortly after. Will that affect how the pages are getting indexed and is there a way we can get Google to go back to the page and try to index the description so it doesn't just appear as a straight link? Thanks, Alex
Technical SEO | | Anti-Alex0 -
Where is Google pulling the date stamp information from?
I found date stamps in front of Google's SERP snippets always very helpful, but where exactly is Google pulling this information from? Some of my sites show dates from past year in the snippets although all of them have a current date stamp on the front page. Would love to make sure that all my sites have show latest date in the snippets.
Technical SEO | | Juergen-1194810