About Google Spider
-
Hello, people!
I have some questions regarding on Google spider.
Many people are saying that "Google spiders only have US IP address."
Is this really true?
But I also saw video from Google's offical blog and it said "Google spider come from all around the world."
At this point I am really confused.
Q1) I researched and it seems like Google spiders have only US IP address. THen what does exactly mean by "Google spider come from all around the world."?
Q2) If Google spider have only US IP address, what happen to site which use IP delivery?
Is this means that Google spider always redirect to us site since they only have US IP?
Can anyone help me to understand??
One more questions! When Google analyzing for cloaking issue, do you think Google analyze when spider crawls the site or after they crawled the site?
-
I think some of the confusion may be due to Google's primarily using IP addresses assigned to their headquarters in Mountain View, California. Google has many (around 20) data centers located outside the US. I recall reading an article whereby at times they used their Mountain View IPs from centers around the world. For security reasons they do not wish the location of all their data centers to be known.
I researched this topic before and I was unable to locate any official information from Google. It would only seem reasonable they crawl from all over the world. If they didn't, then a lot of sites which use geo-based targeting for site navigation would not have most of their content indexed. While it's true a sitemap could be used to overcome the issue, many sites do not use sitemaps and they still get indexed.
-
I do not believe this is true, Google has data centers all over the world in which they crawl from.
Google does not only have spiders crawling from US data centers.
I also have the feeling crawls are based on many factors such as link diversity per region, TLD of domain per region, PR user (still a crawling factor imo) and many more factors.
Overall do not stress Google can crawl from various regions all over the globe, I would be more worried about geo server location and TLD of your domain and also local links.
Kind Regards,
James Norquay.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Old homepage in google on branded keyword
Hello there, I have a strange problem: if you search "probike" (romanian google.ro), i get an OLD homepage with https://www.probike.ro , and my new homepage is https://probike.ro, the real problem is that title/meta description are not available (see attachment). Redirect 301: Ok
Technical SEO | | Shanaki
Robots.txt: Ok
Sitemap: Ok Help, is annoying 🙂 With respect, Andrei I33gThc0 -
Get List Of All Indexed Google Pages
I know how to run site:domain.com but I am looking for software that will put these results into a list and return server status (200, 404, etc). Anyone have any tips?
Technical SEO | | InfinityTechnologySolutions0 -
Google Indexing of Site Map
We recently launched a new site - on June 4th we submitted our site map to google and almost instantly had all 25,000 URL's crawled (yay!). On June 18th, we made some updates to the title & description tags for the majority of pages on our site and added new content to our home page so we submitted a new sitemap. So far the results have been underwhelming and google has indexed a very low number of the updated pages. As a result, only a handful of the new titles and descriptions are showing up on the SERP pages. Any ideas as to why this might be? What are the tricks to having google re-index all of the URLs in a sitemap?
Technical SEO | | Emily_A0 -
Google Structured Data Problem
Hello everyone, About 1-2 weeks ago, I have implemented rich snippets (microdata) for the product pages of my e-commerce site. However, in the web masters tools, google is saying that the crawlers did not detect any structured data in my site. I have also checked my pages using Structured Data Testing Tool. You can see an example test result in the following address. http://www.google.com/webmasters/tools/richsnippets?q=http%3A%2F%2Fwww.tarzimon.com%2Fproduct%2Fnaif-tasarim-torr-aydinlatma-1031 What may cause this problem? Thank you for your help
Technical SEO | | hknkynr0 -
How to Stop Google from Indexing Old Pages
We moved from a .php site to a java site on April 10th. It's almost 2 months later and Google continues to crawl old pages that no longer exist (225,430 Not Found Errors to be exact). These pages no longer exist on the site and there are no internal or external links pointing to these pages. Google has crawled the site since the go live, but continues to try and crawl these pages. What are my next steps?
Technical SEO | | rhoadesjohn0 -
Verification problem with Google Local + Listing
Our company has more than 250+ clients, so I need to open around 250 Gmail account for Google Local + (old Google Places), so for around 250 gmail account I need to verify 250 times the mails with phone. I wanted to know, can I open Google Places with a clients domain mail, example: places@client-domain.com instead of client@gmail.com
Technical SEO | | cvissi0 -
Why did our site drop in Google rankings?
My site's URL (web address) is: http://tinyurl.com/3svn2l9 Hi there, We operate a travel site that lists numerous tours, accommodation and activities. Since 6th August 2011 we have dropped from top 10 SERP rankings of our pages to around result number 100 (page 10) and losing massive amount of visitors via Google Search. Our Yahoo and Bing rankings are still in the top10. We need your advice and quick! The last changes we have made are the following: -redirected the non-www version to the www version on the 1st August -bought advertising with a follow link in a sidebar that is being populated across the site (+4000 pages) about 2 months ago -added a blog to the website 2 weeks ago and posted 2 posts to date. Additionally, our website structure allows visitors (and bots) to see the same listings via different URLs which caused duplicate content. This has been the case since the launch of our website about 1 year ago. To prevent this duplicate content we have placed canonical tags on the individual listings pages. Why did our site all of a sudden plummet in the rankings?
Technical SEO | | Robbern0 -
How to re structure a website & Google
I have a website that requires the site structure to be changed. The website doesnt have many backlnks and rankings are fairly low. I have 11,000 products on the website and want to know the best way to change the site structure without causing 404 errors all over the place. Do I 301 redirect every page? drop all 11,000 pages from the index by adding a no follow no index to all pages? I have the following structure www.domain.co.uk/make/model/part/product I want to change this to www.domain.co.uk/Part/make/model/product whats the best way to preserve the SEO, link juice and on a large scale? 11,000 pages. thank you shivun
Technical SEO | | seohive-2227200