Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
What countries does Google crawl from? Is it only US or do they crawl from Europe and Asia, etc.?
-
Where does Google crawl the web from? Is it in the US only, or do they do it from a European base too? The reason for asking is for GeoIP redirection. For example, if a website is using GeoIP redirection to redirect all US traffic to a .com site and all EU traffic to a .co.uk site, will Google ever see the .co.uk site?
-
Hi Keith,
In my experience Google mainly crawls from the US.
You're quite right, GeoIP redirection can cause major issues with indexation - as if you're redirecting everything from the US to the .com googlebot can't see the .co.uk site.
As such, I'm not a fan. Rather than implementing a hard redirect I prefer Amazon's approach. If you visit the amazon.com site from a UK IP you get a javascript overlay that invites you to visit the .co.uk version of the site instead - they let the user decide which site to view rather than actually redirecting them.
This is a nice solution as it ensures that the search bots can crawl both versions of the site, and rankings aren't endangered.
I hope this helps,
Hannah
-
Keith, I am having the same issue and I agree with you. The fact that Google has data centers in Europe does not necessarily mean the algos are indexing from there. I also want to set Europe and US GeoIP redirection. It would be great to get Mozers opinions on this. Hopefully this post gets freshly reviewed
-
Interesting question - I'd quite like to know what happens here too.
Matt Cutts recently posted a video on cloaking (http://youtu.be/QHtnfOgp65Q) saying that as long as you don't do anything 'special' for Googlebot you're OK, but presumably if you are redirecting IP's based on location and you don't want to prevent Googlebot form accessing your site then effectively you have to do something 'special' for Googlebot (e.g. you're doing one thing for everyone else and a different thing for Googlebot).
-
Hi,
The data center locations is interesting, but it isn't what I was looking for. I need to know whether Google crawls the web from any IP other than US IPs.
To clear up the second question, let me be more specific:
Let's say Google is crawling a .co.uk site from a US IP address. The site is using GEO IP redirection to redirect all US traffic to the .com site. Therefore, when Google attempts to crawl the .co.uk from the US IP address site it will be redirected to the .com site, never seeing the .co.uk site. Can anyone confirm that this is what happens?
-
I found an article from 2008 that shows Google data centre locations: http://bit.ly/mONhf9
Your other question is a bit confusing. Why Google wouldn't see the UK site?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to Localise per Region (Europe, America, APAC, EMEI) and not per country as best SEO practise?
Hi SEO expertises! I am currently working with a client that initially have an English website targeting UK users but want to expand their market into four new regions (Europe, America, APAC and EMEI) keeping English as a main language. I would like to request your help here as I told the client ISO location and hreflang it will be just possible per language and they must need to localise each English region with local keywords, however I would like to double check if it will be any way (Sitemap, Hreflang) we can tell Google we are targeting per region and not per country? Thanks a lot!
International SEO | | Atalig20 -
International SEO - Targeting US and UK markets
Hi folks, i have a client who is based in italy and they set up a site that sells travel experiences in the sout of Italy (the site currently sit on a server in Italy). The site has been set up as gTLDs: www.example.com They only want to target the US and the UK market to promote their travel experiences and the site has only the english version (the site does not currently offer an italian version). If they decide to go for the gTLDs and not actually change to a ccTLDs (which would be ideal from my point of view) how are the steps to be taken to set this up correctly on GSC? They currently only have one property registered on GSC: www.exapmple.com therefore i guess the next steps are: Add new property - www.example.com/uk and and set up geo targeting for UK Existing property - www.example.com/ set up geo targeting for US In case the client does not have the budget to optimise the content for american and british languages, would still make sense to have 2 separate property in GSC (example.com for US market and example.com/uk for UK market)? Few considerations: Add canonical tag to avoid duplicate content across the two versions of the site (in the event there is no budget to optimise the content for US and UK market)? Thank you all in advance for looking into this David
International SEO | | Davide19840 -
Country and Language Specific URL Paths
Wanted to ask everyone a questions: So our company is going to be doing a website that is going to be full of videos. The url path will be country.domain.com/language/slug/content-id. We redirect the user when they go to the different country. So if you're in spain on a train to france your URL will change from es.domain.com/es/slug/content-id to fr.domain.com/es/slug/content-id. Each country can listen to each video in all languages. My question is with hreflang tags and canonicals. Aside from targeting users in a certain country via Google Search Console, how do I eliminate duplication and tell Google which I'd like to show up via which country. In spain I would like es.domain.com/es/slug/content-id to show in Google and would have hreflang tags on each of the es.domain pages but what about fr.domain.com/es/slug/content-id since it would show the same content? I can't canonical to one of them since I need them to show in their respective country. How do I show the difference in language and country without showing duplication?
International SEO | | mattdinbrooklyn0 -
Targeting Countries in the Middle East
Hi guys, I have a client based in the Middle East using a generic top level domain (.com), and they want to target multiple countries in the GCC (UAE, Saudi Arabia, Kuwait, Qatar etc). I’m thinking that using the hreflang tag would be the best solution here, however the pages will mostly have the exact same content. There will only be slight changes on some pages in terms of using localised title tags [client service] followed by [targeted country], h1's and meta descriptions. Is this the correct approach? And if so should this be implemented side wide or can it be implemented on selected pages only? The site will be in English only.
International SEO | | Jbeetle0 -
Google.ie returning more and more UK based results, why?
I have discovered the most infuriating issue with Google Search for Irish users and it seems to be getting increasingly worse in the last 2 years or so. This is not only frustrating as a business owner (in fact it could bring a business to its knees) but it is rage inducing as a consumer.
International SEO | | Secrets
Google knows the location where I am searching from and I'm using google.ie yet I still get just a small number of Irish websites usually followed by eBay and Amazon results then a never ending list of websites that are based in the United Kingdom. Now, I know the one thing that we all have in common is the use of the English language, however what we don't have in common is shipping costs. In order to slightly increase the number of Irish based companies I need to add in the phrase 'Ireland' to my search (on google.ie in Ireland) and this makes only a small difference. In fact, oftentimes Google seems to throw in the odd American or Australian site just to really wind me up.
It's completely absurd that Google rarely returns results for .ie websites or irish based websites when searching in Ireland. Many UK companies don't ship to Ireland (including many of the eBay and Amazon results). This is killing Irish businesses who have the products and cheaper or free shipping and many how are working damn hard on their SEO are still being passed up for companies that have nothing to do with our economy.... Why oh why is this happening.0 -
How To Rank A UK Website On Google.com (US)
Hi, I've done some research on this but couldn't find any definitive answer I can trust! We have a client who resides in the UK. They have '.com' domain, hosted on a UK server, using UK spelling. Their business objective for this year is to expand in the USA, including the opening of a warehouse over there. They are wanting us to rank their website on both Google.co.uk and Google.com (North America); besides changing the geolocation settings in GWT's, and building links from .com websites is there anything else we can do to increase their visibility on Google.com? Many thanks in advance, appreciated!
International SEO | | Webpresence
Lee.0 -
Geographic Target set up in Google webmaster tool
Hi, When i launched my web site 3 months ago ( I'm am very new to SEO) I have set up the geographical target section in Google webmaster tool for US. Now, I'm thinking to change it to some other geo target to see if i can get more traffic. However, recently few of my prompted keywords got really well in Google US ranking. Here are my Questions: if i will change the geo settings in webmaster tool will effect the ranking i already managed to achieve in US? In the list of all the countries in Google webmaster tools what does is mean "unlisted"? Can i select more than one country to target and if I can how? Thanks!! Raviv
International SEO | | Indiatravelz0 -
Google US vs Google UK
I could have posted this somewhere else, but I cannot find it. So, I have keywords that rank well in Google US and many that do well in Google UK too. I thought all of my keywords ranking well in the US would also rank well the UK. I have figured out today that it is not the case. Why would I rank in the top 3 in the US and not even show up in the top 50 in the UK? It is very strange. Thanks for your help! I am not super new to SEO or web business. I have had a very good company that has been ranking well since 2004.
International SEO | | journeybeyondtravel0