What countries does Google crawl from? Is it only US or do they crawl from Europe and Asia, etc.?
-
Where does Google crawl the web from? Is it in the US only, or do they do it from a European base too? The reason for asking is for GeoIP redirection. For example, if a website is using GeoIP redirection to redirect all US traffic to a .com site and all EU traffic to a .co.uk site, will Google ever see the .co.uk site?
-
Hi Keith,
In my experience Google mainly crawls from the US.
You're quite right, GeoIP redirection can cause major issues with indexation - as if you're redirecting everything from the US to the .com googlebot can't see the .co.uk site.
As such, I'm not a fan. Rather than implementing a hard redirect I prefer Amazon's approach. If you visit the amazon.com site from a UK IP you get a javascript overlay that invites you to visit the .co.uk version of the site instead - they let the user decide which site to view rather than actually redirecting them.
This is a nice solution as it ensures that the search bots can crawl both versions of the site, and rankings aren't endangered.
I hope this helps,
Hannah
-
Keith, I am having the same issue and I agree with you. The fact that Google has data centers in Europe does not necessarily mean the algos are indexing from there. I also want to set Europe and US GeoIP redirection. It would be great to get Mozers opinions on this. Hopefully this post gets freshly reviewed
-
Interesting question - I'd quite like to know what happens here too.
Matt Cutts recently posted a video on cloaking (http://youtu.be/QHtnfOgp65Q) saying that as long as you don't do anything 'special' for Googlebot you're OK, but presumably if you are redirecting IP's based on location and you don't want to prevent Googlebot form accessing your site then effectively you have to do something 'special' for Googlebot (e.g. you're doing one thing for everyone else and a different thing for Googlebot).
-
Hi,
The data center locations is interesting, but it isn't what I was looking for. I need to know whether Google crawls the web from any IP other than US IPs.
To clear up the second question, let me be more specific:
Let's say Google is crawling a .co.uk site from a US IP address. The site is using GEO IP redirection to redirect all US traffic to the .com site. Therefore, when Google attempts to crawl the .co.uk from the US IP address site it will be redirected to the .com site, never seeing the .co.uk site. Can anyone confirm that this is what happens?
-
I found an article from 2008 that shows Google data centre locations: http://bit.ly/mONhf9
Your other question is a bit confusing. Why Google wouldn't see the UK site?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Problem to get multilingual posts indexed on Google
Last year on June I decided to make my site multi-lingual. The domain is: https://www.dailyblogprofits.com/ The main language English and I added Portuguese and a few posts on Spanish. What happened since then? I started losing traffic from Google and posts on Portuguese are not being indexed. I use WPML plugin to make it multi-lingual and I had Yoast installed. This week I uninstalled Yoast and when I type on google "site:site:dailyblogprofits.com/pt-br" I started seeing Google indexing images, but still not the missing posts. I have around 145 posts on Portuguese, but on Search Console it show only 57 hreflang tags. Any idea what is the problem? I'm willing to pay for an SEO Expert to resolve this problem to me.
International SEO | | Cleber0090 -
Setting up I.P Filter Google Analytics - I.p ending with 0/24
Hi everyone, Your help would be much appreciated for the following: I am trying to setup I.P filters for our Google Analytic account to exclude internal traffic. We are located in multiple locations and each location have multiple I.p addresses. The I.P addresses we have end either by 0/24 which apparently means they provide a range from 0 to 255 and or 128/25. I have tried to setup the I.P addresses in different formats on the GA filter but they are apparently are not valid: example of one setup I tried: 1**.\2**.\8*.([0-256]) I have gone through the Filter setup guide from Google but I must be doing something wrong- probably to do on how I setup the I.P's ending with 0/24 and 128/25 If anyone could help me on how I can set up the I.P filters Google analytic would be great. The I.P addresses look like the following (changed digits): Location 1: 174.177.179.0/25 174.177.179.128/25 Location 2: 196.222.87.0/24
International SEO | | AlphaDigital2
194.59.197.0/24 Thanks you so much for your help, L.0 -
How to set up international SEO for english speaking countries
Hi, My company have offices around the world. However they also provide different services and products depending on the region. For example our offices in the USA, UK and Australia all provide different services to each other. My question is, how do I set up my WordPress website up to cater for these different countries and services? I think the simple answer would be to build a separate website for each, but this would be too costly and we don't have the resources to maintain all three. Many thanks for your time, Tom
International SEO | | CoGri0 -
Google Analytics & Webmaster Tools Filtering
I've just set up a client who uses internationalization on Google Webmaster tools and Google Analytics. For easier management they opted to use subfolders rather than subdomains or cctlds. So I set up a Google Analytic Property, with one Unfiltered profile, and another 3 profiles filtered per language. With the main language English being exclude anything starting /fr/ /de/ as it resides on root. The filters seem to work fine; however after linking this to the Google Webmaster Account to be able to access Search Engine Optimization I do not seem to get any language filtered data. I was wondering if someone had any idea or possible solution to this problem. As I would expect to at least have the Landing Pages if not exactly the keywords filtered by the same criteria that the rest of the data is being filtered. I know there's also an option to create a separate webmaster tools account, however this way I still cannot filter just the English; and I cannot link it to all the separate profiles.
International SEO | | jonmifsud0 -
Help with .asia domain extension
hi there i m planning to buy a domain with .asia extension. I want to know ....values of these tlds regarding seo.did i chose wrong tlds?? purely targeting emd traffic. One of my site with .biz for a emd in "biz op" ranked less than 2 weeks ..now i have 200-300 traffic daily. i asked here because of i have no experience with .asia extension. 1.can i target the audience demographically with these .asia for u.s. , u.k .or brazil. 2. these extension can be good for seo ? can i ranked high in serp for low ocmpetition terms any suggestion or idea & tips for me??? thanks in advances.
International SEO | | ranktrack0 -
Same language manage many country
Hello, I would like to hear how you would handle the following situation. I make website for people with a .be domain that also want to score in .nl . Both countries speak dutch. There are however slight variations between the two countries. For some it does not matter for others it can be handy. Ex. A specific product might be called diffrent in one country then the other. The main problem is for those that have a .be domain will not score on a .nl domain and visa versa. This due geo targeting. What would you do? Imagine we already have a .be domain (.be == belgium, .nl==Netherland both speak dutch) A: Buy a .nl and copy and show the same content on .nl as .be
International SEO | | nono_1974
B: Buy a .nl and make new content (lots of work)
😄 Buy a .EU, .com adres and Geotarget nl.somedomain.eu , be.somedomain.eu . But what about the content The main problem is double content and targeting the two countries.0 -
Google Webmaster Tools - Geographic target - Time till change
One of my clients is targeting UK customers. They have UK IP address. I then noticed that they had their Geographic target set to "Target users in: United States". I advised them to change this. Anyone know: How this could have been set, the client says they didn't do it? How long will it take for Google to see this has been changed? Thanks in advance. Justin
International SEO | | GrouchyKids0 -
Www.google.rs for SERBIA is mising in Engienes
Why you dont have SERBIA in engines for google - www.google.rs You have Croatia, Bosnia and Hercegovina and Slovenia and Serbia is missing 😞 Best regards kasa
International SEO | | stamparija2