About Google Spider
-
Hello, people!
I have some questions regarding on Google spider.
Many people are saying that "Google spiders only have US IP address."
Is this really true?
But I also saw video from Google's offical blog and it said "Google spider come from all around the world."
At this point I am really confused.
Q1) I researched and it seems like Google spiders have only US IP address. THen what does exactly mean by "Google spider come from all around the world."?
Q2) If Google spider have only US IP address, what happen to site which use IP delivery?
Is this means that Google spider always redirect to us site since they only have US IP?
Can anyone help me to understand??
One more questions! When Google analyzing for cloaking issue, do you think Google analyze when spider crawls the site or after they crawled the site?
-
I think some of the confusion may be due to Google's primarily using IP addresses assigned to their headquarters in Mountain View, California. Google has many (around 20) data centers located outside the US. I recall reading an article whereby at times they used their Mountain View IPs from centers around the world. For security reasons they do not wish the location of all their data centers to be known.
I researched this topic before and I was unable to locate any official information from Google. It would only seem reasonable they crawl from all over the world. If they didn't, then a lot of sites which use geo-based targeting for site navigation would not have most of their content indexed. While it's true a sitemap could be used to overcome the issue, many sites do not use sitemaps and they still get indexed.
-
I do not believe this is true, Google has data centers all over the world in which they crawl from.
Google does not only have spiders crawling from US data centers.
I also have the feeling crawls are based on many factors such as link diversity per region, TLD of domain per region, PR user (still a crawling factor imo) and many more factors.
Overall do not stress Google can crawl from various regions all over the globe, I would be more worried about geo server location and TLD of your domain and also local links.
Kind Regards,
James Norquay.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google is indexing our old domain
We changed our primary domain from vivitecsolutions.com to vivitec.net. Google is indexing our new domain, but still has our old domain indexed too. The problem is that the old site is timing out because of the https: Thought on how to make the old indexing go away or properly forward the https?
Technical SEO | | AdsposureDev0 -
Not all images indexed in Google
Hi all, Recently, got an unusual issue with images in Google index. We have more than 1,500 images in our sitemap, but according to Search Console only 273 of those are indexed. If I check Google image search directly, I find more images in index, but still not all of them. For example this post has 28 images and only 17 are indexed in Google image. This is happening to other posts as well. Checked all possible reasons (missing alt, image as background, file size, fetch and render in Search Console), but none of these are relevant in our case. So, everything looks fine, but not all images are in index. Any ideas on this issue? Your feedback is much appreciated, thanks
Technical SEO | | flo_seo1 -
Pages are Indexed but not Cached by Google. Why?
Hello, We have magento 2 extensions website mageants.com since 1 years google every 15 days cached my all pages but suddenly last 15 days my websites pages not cached by google showing me 404 error so go search console check error but din't find any error so I have cached manually fetch and render but still most of pages have same 404 error example page : - https://www.mageants.com/free-gift-for-magento-2.html error :- http://webcache.googleusercontent.com/search?q=cache%3Ahttps%3A%2F%2Fwww.mageants.com%2Ffree-gift-for-magento-2.html&rlz=1C1CHBD_enIN803IN804&oq=cache%3Ahttps%3A%2F%2Fwww.mageants.com%2Ffree-gift-for-magento-2.html&aqs=chrome..69i57j69i58.1569j0j4&sourceid=chrome&ie=UTF-8 so have any one solutions for this issues
Technical SEO | | vikrantrathore0 -
CDN Being Crawled and Indexed by Google
I'm doing a SEO site audit, and I've discovered that the site uses a Content Delivery Network (CDN) that's being crawled and indexed by Google. There are two sub-domains from the CDN that are being crawled and indexed. A small number of organic search visitors have come through these two sub domains. So the CDN based content is out-ranking the root domain, in a small number of cases. It's a huge duplicate content issue (tens of thousands of URLs being crawled) - what's the best way to prevent the crawling and indexing of a CDN like this? Exclude via robots.txt? Additionally, the use of relative canonical tags (instead of absolute) appear to be contributing to this problem as well. As I understand it, these canonical tags are telling the SEs that each sub domain is the "home" of the content/URL. Thanks! Scott
Technical SEO | | Scott-Thomas0 -
Google autorship in specific field?
Hi, I want to ask you about something I 've read about google and authorship. It is written that it is better to show yourself as a author in a specific field. I myself have knowledge and interest in many fields - like SEO, vegan living, martial arts. And I want to be seen as specialist in all of them. Does it mean that we are limited to mark with autorship articles in only one field, in order to be seen as expert in a specific field? f.e. Should I mark with "rel=author" the articles that are about SEO because I want to be seen as author in that specific field for sure. Iif I mark with "rel=author" articles also about martial arts would these affect the understanding about my expertise in SEO?
Technical SEO | | vladokan0 -
Google Not liking Magento Sites?
Hello, I'm new to the community and I wonder if anyone can help us shed a light on this SEO issue we are having. We have 3 magento websites that is being affected. Whats happening is that those site were ranked for a specific keyword for few months, but all of a sudden, it just drop like crazy. It went from top 10 to about 150 in a bout a weeks period. Some site, it's not even ranked or stopped ranking and visible on the search engine. Is google not liking MAgento for some reason?? Any help or suggestions will be appreciated! thanks
Technical SEO | | solution.advisor0 -
Why Can't I Get on Google?
I've employed many of the suggestions of SEOMoz and getting a Grade "A" on a particular keyword. I'm now #4 on Yahoo and Bing. However, my site hasn't cracked the top 50 in Google. Why? I see a similar pattern with other keywords, many on yahoo and bing but only a few of my subpages get #45-48 on Google. Any ideas? http://www.gospelebooks.net
Technical SEO | | mrjgardiner0 -
Sitemap.xml problem in Google webmaster
Hi, My sitemap.xml is not submitting correctly in Google Webmaster. There is 697 url submitted but only 56 are in Google index. At the top of webmaster this is what it says ->>> http://www.example.com/sitemap.xml has been resubmitted. But when when I clicked status button RED X occurs. Any suggestions about this, thanks...
Technical SEO | | Socialdude0