What countries does Google crawl from? Is it only US or do they crawl from Europe and Asia, etc.?
-
Where does Google crawl the web from? Is it in the US only, or do they do it from a European base too? The reason for asking is for GeoIP redirection. For example, if a website is using GeoIP redirection to redirect all US traffic to a .com site and all EU traffic to a .co.uk site, will Google ever see the .co.uk site?
-
Hi Keith,
In my experience Google mainly crawls from the US.
You're quite right, GeoIP redirection can cause major issues with indexation - as if you're redirecting everything from the US to the .com googlebot can't see the .co.uk site.
As such, I'm not a fan. Rather than implementing a hard redirect I prefer Amazon's approach. If you visit the amazon.com site from a UK IP you get a javascript overlay that invites you to visit the .co.uk version of the site instead - they let the user decide which site to view rather than actually redirecting them.
This is a nice solution as it ensures that the search bots can crawl both versions of the site, and rankings aren't endangered.
I hope this helps,
Hannah
-
Keith, I am having the same issue and I agree with you. The fact that Google has data centers in Europe does not necessarily mean the algos are indexing from there. I also want to set Europe and US GeoIP redirection. It would be great to get Mozers opinions on this. Hopefully this post gets freshly reviewed
-
Interesting question - I'd quite like to know what happens here too.
Matt Cutts recently posted a video on cloaking (http://youtu.be/QHtnfOgp65Q) saying that as long as you don't do anything 'special' for Googlebot you're OK, but presumably if you are redirecting IP's based on location and you don't want to prevent Googlebot form accessing your site then effectively you have to do something 'special' for Googlebot (e.g. you're doing one thing for everyone else and a different thing for Googlebot).
-
Hi,
The data center locations is interesting, but it isn't what I was looking for. I need to know whether Google crawls the web from any IP other than US IPs.
To clear up the second question, let me be more specific:
Let's say Google is crawling a .co.uk site from a US IP address. The site is using GEO IP redirection to redirect all US traffic to the .com site. Therefore, when Google attempts to crawl the .co.uk from the US IP address site it will be redirected to the .com site, never seeing the .co.uk site. Can anyone confirm that this is what happens?
-
I found an article from 2008 that shows Google data centre locations: http://bit.ly/mONhf9
Your other question is a bit confusing. Why Google wouldn't see the UK site?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is there a Risk Around Creating a Website for Each Country in The World?
Is there a risk around creating a website for each country in the world with similar to identical content depending on the language? We need to serve prices and the local currencies and be compliant with regulations. We're planning to use rel=canonical and HREFLANG tags to help with consolidation and GEO-targeting.
International SEO | | ari_seo0 -
Opening Up Webmaster Tools Geo-Targeting to Everyone Instead of US
Hi Mozzers, I wanted to get everyone's opinion on this. We have an e-commerce site that was targeting 3 different geo-locations. The main one is a .com. The other two were Canada and UK based. We recently turned off our Canada site and 301 to the .com. The .com site had been targeted towards United States in Google Webmaster Tools. Should I remove the Webmaster Tools geo-targeting now that Canada is pointing towards our main site? I don't want to lose rankings for US since that is our main market and the .com site is our main revenue driver and focus. I'm inclined to leave it as is because I still see our .com site ranking when I search in Google CA. From what I've seen is that it can take months for something to change and I definitely don't want anything to go awry. Just wanted to get everyone's thoughts on this. Thanks in advance.
International SEO | | SylviaH0 -
Can you target the same site with multiple country HREFlang entries?
Hi, I have a question regarding the country targeting aspect of HREFLANG. Can the same site be targeted with multiple country HREFlang entries? Example: A global company has an English South African site (geotargeted in webmaster tools to South Africa), with a hreflang entry targeted to "en-za", to signify English language and South Africa as the country. Could you add entries to the same site to target other English speaking South African countries? Entries would look something like this: (cd = Congo, a completely random example) etc... Since you can only geo-target a site to one country in WMT would this be a viable option? Thanks in advance for any help! Vince
International SEO | | SimonByrneIFS0 -
How To Rank A UK Website On Google.com (US)
Hi, I've done some research on this but couldn't find any definitive answer I can trust! We have a client who resides in the UK. They have '.com' domain, hosted on a UK server, using UK spelling. Their business objective for this year is to expand in the USA, including the opening of a warehouse over there. They are wanting us to rank their website on both Google.co.uk and Google.com (North America); besides changing the geolocation settings in GWT's, and building links from .com websites is there anything else we can do to increase their visibility on Google.com? Many thanks in advance, appreciated!
International SEO | | Webpresence
Lee.0 -
What is best practice of using google translate
Hi, I'm thinking of adding google translate to our retailing site so that we could reach more international customers. what about the pros&cons? Any experience of success of utilising it and what potential issues should I be looking at? Thanks
International SEO | | LauraHT0 -
Ranking UK company in Google.com
Hi all, I have a UK client with a .com domain, hosted on a US server, but the physical business premises is based in the UK. Their product is a really great product and available for export to the US. I want to rank them higher in the US, more specifically Google.com. I've helped them rank very well organically in the UK (google.co.uk) for some great terms, however they rank almost nowhere in google.com (gl=us) for the same terms, for example: In Google.co.uk they rank #3 for the key-phrase.
International SEO | | seowoody
In Google.com they rank #90 for the same key-phrase. I've got them some great US focused links with PR coverage including MSN Cars, nydailynews.com etc. I just wondered if there was any one "golden ticket" for boosting US rankings? I've read that a physical business premises located in the US helps a lot. Can anyone confirm this and if so, would a rented PO box in the US help? The site has great social signals too, growing twitter following and many FB likes/shares etc. Any other tips/advice? Thanks in advance,
Cheers,
Woody 🙂0 -
Google Places and Hotel Finder
I work with a few hotels in Costa Rica and Nicaragua. Google Places has not launched in these regions so it has been impossible for me to start an account and get my properties listed in hotel finder. Somehow my competition has managed to get one of their properties listed and it's killing me! Does anyone know of any way to speed up the process? I'm very involved with Google+, FB, Twitter, Pinterest but G Places and Hotel finder or Google Maps don't want to pick me up. Any ideas? Thanks Mozzers. aSrDvhr
International SEO | | janders0 -
How can I replicate India SERPs from US
I was recently checking out our analytics keywords stats and I am seeing traffic noted as google organic traffic coming from India. When I do a search on http://www.google.co.in/ for those terms using a "incognito" browser with no cookies, and not logged in I don't see our site in the first 100 results. The same thing happens on google.com if I try to set the account in india. I am trying to see if this might be an issue with an adwords campaign not getting auto-tagged (auto-tagging is turned on in adwords), or if there is something else going on. The keyword campaign gets many more clicks than analytics shows for this organic option so perhaps it is an odd rare occurrence. I am glad to have the traffic, but I want to be sure it is actually tracking correctly as organic, or if PPC is not tagging for some reason 😉 Any suggestions or thoughts?
International SEO | | SL_SEM0