What countries does Google crawl from? Is it only US or do they crawl from Europe and Asia, etc.?
-
Where does Google crawl the web from? Is it in the US only, or do they do it from a European base too? The reason for asking is for GeoIP redirection. For example, if a website is using GeoIP redirection to redirect all US traffic to a .com site and all EU traffic to a .co.uk site, will Google ever see the .co.uk site?
-
Hi Keith,
In my experience Google mainly crawls from the US.
You're quite right, GeoIP redirection can cause major issues with indexation - as if you're redirecting everything from the US to the .com googlebot can't see the .co.uk site.
As such, I'm not a fan. Rather than implementing a hard redirect I prefer Amazon's approach. If you visit the amazon.com site from a UK IP you get a javascript overlay that invites you to visit the .co.uk version of the site instead - they let the user decide which site to view rather than actually redirecting them.
This is a nice solution as it ensures that the search bots can crawl both versions of the site, and rankings aren't endangered.
I hope this helps,
Hannah
-
Keith, I am having the same issue and I agree with you. The fact that Google has data centers in Europe does not necessarily mean the algos are indexing from there. I also want to set Europe and US GeoIP redirection. It would be great to get Mozers opinions on this. Hopefully this post gets freshly reviewed
-
Interesting question - I'd quite like to know what happens here too.
Matt Cutts recently posted a video on cloaking (http://youtu.be/QHtnfOgp65Q) saying that as long as you don't do anything 'special' for Googlebot you're OK, but presumably if you are redirecting IP's based on location and you don't want to prevent Googlebot form accessing your site then effectively you have to do something 'special' for Googlebot (e.g. you're doing one thing for everyone else and a different thing for Googlebot).
-
Hi,
The data center locations is interesting, but it isn't what I was looking for. I need to know whether Google crawls the web from any IP other than US IPs.
To clear up the second question, let me be more specific:
Let's say Google is crawling a .co.uk site from a US IP address. The site is using GEO IP redirection to redirect all US traffic to the .com site. Therefore, when Google attempts to crawl the .co.uk from the US IP address site it will be redirected to the .com site, never seeing the .co.uk site. Can anyone confirm that this is what happens?
-
I found an article from 2008 that shows Google data centre locations: http://bit.ly/mONhf9
Your other question is a bit confusing. Why Google wouldn't see the UK site?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to not appear in incorrect country
Hi! I have a problem with the results in Google. My website ranks, but all countries appear in the search for example from Argentina.
International SEO | | SEO-Mediabros
I have the correct hreflang tags. How can I "block" the results of other countries? This is the site and this is the search example. Thank you! 🙂1 -
International SEO - Targeting US and UK markets
Hi folks, i have a client who is based in italy and they set up a site that sells travel experiences in the sout of Italy (the site currently sit on a server in Italy). The site has been set up as gTLDs: www.example.com They only want to target the US and the UK market to promote their travel experiences and the site has only the english version (the site does not currently offer an italian version). If they decide to go for the gTLDs and not actually change to a ccTLDs (which would be ideal from my point of view) how are the steps to be taken to set this up correctly on GSC? They currently only have one property registered on GSC: www.exapmple.com therefore i guess the next steps are: Add new property - www.example.com/uk and and set up geo targeting for UK Existing property - www.example.com/ set up geo targeting for US In case the client does not have the budget to optimise the content for american and british languages, would still make sense to have 2 separate property in GSC (example.com for US market and example.com/uk for UK market)? Few considerations: Add canonical tag to avoid duplicate content across the two versions of the site (in the event there is no budget to optimise the content for US and UK market)? Thank you all in advance for looking into this David
International SEO | | Davide19840 -
Country and Language Specific URL Paths
Wanted to ask everyone a questions: So our company is going to be doing a website that is going to be full of videos. The url path will be country.domain.com/language/slug/content-id. We redirect the user when they go to the different country. So if you're in spain on a train to france your URL will change from es.domain.com/es/slug/content-id to fr.domain.com/es/slug/content-id. Each country can listen to each video in all languages. My question is with hreflang tags and canonicals. Aside from targeting users in a certain country via Google Search Console, how do I eliminate duplication and tell Google which I'd like to show up via which country. In spain I would like es.domain.com/es/slug/content-id to show in Google and would have hreflang tags on each of the es.domain pages but what about fr.domain.com/es/slug/content-id since it would show the same content? I can't canonical to one of them since I need them to show in their respective country. How do I show the difference in language and country without showing duplication?
International SEO | | mattdinbrooklyn0 -
How can I change the currency Google lists my products with in the SERP?
I.e. This product - http://www.absoluteautomation.ca/fgd400-sensaphone400-p/fgd400.htm - shows up as USD in the SERP. (In the US it just won't show a currency, if Canada it will show USD on the SERP). My pricing is all in CAD, how can I tell Google this? (It knows pricing is CAD in my Google Product Listings/Merchant Center). Thanks!
International SEO | | absoauto0 -
Mini Google Search box within organic sitelinks
I'm trying to find solid information on Google's criteria on showing a "mini Google search box" within sitelinks? Example: if you type-in "wikipedia" on Google.com, a mini search box appears within the site links. This is for organic listing not paid listing. The Google Support page on Sitelinks does not state any info on the mini google search box. http://support.google.com/webmasters/bin/answer.py?hl=en&answer=47334
International SEO | | Mobies0 -
Google Webmaster Tools - Geographic target - Time till change
One of my clients is targeting UK customers. They have UK IP address. I then noticed that they had their Geographic target set to "Target users in: United States". I advised them to change this. Anyone know: How this could have been set, the client says they didn't do it? How long will it take for Google to see this has been changed? Thanks in advance. Justin
International SEO | | GrouchyKids0 -
Same website in different countries, best practices for SEO?
Hey Guys, I have read several similar questions regarding mine, but none seem to truly cover my question. Basically, we have a company named Junair. We created the website for the company here in Australia (http://www.junair.com.au). As can be seen throughout the page, it mentions that it caters for both Australia and NZ (NZ has its own phone number). It does ok in the rankings at the moment, but rankings will continue to rise in the future once more links are getting picked up. Now however, the Junair team in NZ purchased the NZ domain http://www.junair.co.nz and redirected it to the Australian page. No matter which page you visit on the NZ URL, the URL will never change, and neither will the page title. They have now contacted us and asked to perform SEO on the NZ domain so the NZ domain would show up in searches on Google NZ. At the moment, when searching for "Junair" on google.co.nz, the Australian domain is coming up. How could I change this so the NZ URL would show instead? And what would be the best practices to perform SEO on the NZ URL, should I just create links pointing to http://www.junair.co.nz ? Thank you, Roderic
International SEO | | Michael-Goode0 -
Ranking well in Google US but not showing up in UK
Hi, We are trying to position a specific site for a big sport event that will be played at the end of May. We only care about Google UK. It had no relevant content until last week. The site is a almost a year old but we just started to get links for it (all from the UK). They worked so fast: we were in page 2 of SERPs in the UK within a few days. But suddenly the site has disappeared and we're not even in top 100 anymore. However, we are ranking extremely well in the US (first page for the keywords we wanted to rank for). The site is a .com and it's hosted in Wordpress.com (with a custom URL). I understand that that can be a problem, but we have already told Google that our business is in the UK through Webmaster Tools. I guess that with time and more UK links, Google would ultimately understand that our focus is the UK audience, but unfortunately we have no time and we can't wait due to the date of the sport event. What do you think that we can do to rank well in the UK and not in the US as fast as possible? Thanks!
International SEO | | jorgediaz0