International Sites - Sitemaps, Robots & Geolocating in WMT
-
Hi Guys,
I have a site that has now been launched in the US having originally just been UK. In order to accommodate this, the website has been set-up using directories for each country. Example:
As the site was originally set-up for UK, the sitemap, robots file & Webmaster Tools account were added to the main domain. Example:
The question is does this now need changing to make it specific for each country. Example:
The sitemap and robots.txt for the UK would move to:
and the US would have its own separate sitemap and robots.txt. Example :
Also in order to Geolocate this in WMT would this need to be done for each directory version instead of the main domain? Currently the WMT account for the UK site is verified at www.domain.com, would this need reverifying at domain.com/en-gb?
Any help would be appreciated! Thanks!
-
Thanks for the insights. Very helpful. What about the robots.txt, though? Should it stay under http://www.example.com, where crawlers can find the file?
-
Yes, that's how I did it.
-
Thanks for the response, much appreciated.
Would this mean that the sitemap featured at www.domain.com would need to reference each directory version (UK & US etc etc), leaving the en-gb & en-us to reference just their own?
-
Hi, here's what Google officially has to say:
Webmaster Tools data and reporting work best on a site level. For example, if your site http://www.example.com has separate sections for different countries, we recommend adding each of those subsites or subfolders as a separate site. For example, if you have a travel site with specific subfolders covering Ireland, France, and Spain, you could add the following sites to your Webmaster Tools account:
- http://www.example.com
- http://www.example.com/france
- http://www.example.com/ireland
- http://www.example.com/spain
Following this I have solved the exactly the same problem you had for a client. And it works perfectly.
- _The question is does this ["sitemap"] now need changing to make it specific for each country. _What I have done is add the sitemaps for each subdirectory as well as the main domain. It works (tracking and all that) and is easy to do.
- _Also in order to Geolocate this in WMT would this need to be done for each directory version instead of the main domain? _Yes, that is the way to go.
- _Currently the WMT account for the UK site is verified at www.domain.com, would this need reverifying at domain.com/en-gb? _Yes, it does. And so do every other subdirectory you add down the line. That allows precise international targeting per subdirectory via the hreflang and country targeting settings and setting the main GWT site for the root domain to be for global traffic.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Correct Hreflang & Canonical Tags for Multi-Regional Website English Language Only having URL Parameters
Dear friends, We have a multi-regional website in English language only having the country selector on the top of each page and it adds countrycode parameters on each url. Website is built in Magento 1.8 and having 1 store with multiple store views. There is no default store set in Magento as I discussed with developer. Content is same for all the countries and only currency is changed. In navigation there are urls without url parameters but when we change store from any page it add parameters in the url for same page hence there are total 7 URLs. 6 URLs for each page (with country parameters) and 1 master url (without parameters) and making content duplicity. We have implemented hreflang tags on each page with url parameters but for canonical we have implemented master page url as per navigation without url parameters Example on this page. I think this is correct for master page but we should use URL parameters in canonical tags for each counry url too and there should be only 1 canonical tag on each country page url. Currently all the country urls are having master page canoncial tag as per the example. Please correct me if I am wrong and **in this case what has to be done for master page? **as google is indexing the pages without parameters too. We are also using GEOIP redirection for each store with country IP detection and for rest of the countries which are not listed on the website we are redirecting to USA store. Earlier it was 301 but we changed it to 302. Hreflang tags are showing errors in SEMRush due to redirection but in GWT it's OK for some pages it's showing no return tags only. Should I use **x-default tags for hreflang and country selector only on home page like this or should I remove the redirection? **However some of the website like this using redirection but header check tool doesn't show the redirection for this and for our website it shows 302 redirection. Sorry for the long post but looking for your support, please.
International SEO | | spjain810 -
MultiRegional site indexing problems
Hello there!!! I have a multiregional site and dealing with some indexing problems. The problem is that google have only indexed our USA site We have: -set up hreflang tags -set up specific subdirectories https://www.website.com/ (en-us site and our main site) https://www.website.com/en-gb https://www.website.com/en-ca https://www.website.com/fr-ca https://www.website.com/fr-fr https://www.website.com/es-es ..... -set up automatic GEO IP redirects (301 redirects) -created a sitemap index and a different sitemap for each regional site -created a google webmaster's tool for each country targeted -created translations for each different language and added some canonicals to the US' site when using English content. The problem is that Google is not indexing our regional sites. I think that the problem is that google is using a US bot when spidering the site, so it will be always redirect to the US version by a 301 redirect. I have used fetch as google with some of our regional folders and asked for "Indexing requested for URL and linked pages", but still waiting. Some ideas?? changing 301 to 302? Really don't know what to do. Thank you so much!!
International SEO | | Alejandrodurn0 -
Splitting a site into 2 international sites
Hi all, I have a client that currently has a .com domain that ranks in both the US and the UK for various search terms. They have identified a need to provide different information for UK and US visitors which will require 2 versions of all pages. If we set up a .co.uk domain and keep the .com obviously that will be a brand new UK site which will have zero rankings. Any suggestions as to the best way to introduce this second version of the content without losing UK rankings? Thanks
International SEO | | danfrost0 -
International SEO errors
Hello, In a muddle here. A website has a .co.uk and a .com version. They want to target the UK market and the USA market respectively. The content for the UK version has been localised for the UK audience (e.g. spellings etc) but the content is the same in both sites. There are errors in .co.uk version in webmaster tools : International Targeting | Language > 'en' - no return tags__URLs for your site and alternate URLs in 'en' that do not have return tags.**Q 1) What does this mean?**I can see that both the .com and .co.uk version has only this in place:**2) Should they actually have respectively?**3) Do they also need rel=canonical from the .co.uk to the .comAny help would be appreciated.
International SEO | | AL123al0 -
Include mobile and international versions of pages to sitemap or not?
My pages already have alternate and hreflang references to point to international and mobile versions of the content. If I add 5 language desktop versions and 5 language mobile versions as https://support.google.com/webmasters/answer/2620865?hl=en explains, my sitemap will get bulky. What are the pros and cons for referencing all page versions in sitemap and for include just general (English/Desktop) version in sitemap?
International SEO | | poiseo0 -
Shall I automatically redirect international visitors from www.domain.com to e.g. www.domain.com/es? What is best SEO practice?
We have chosen the one domain approach with our international site having different language versions in subdirectory of main domain:
International SEO | | lcourse
www.domain.com/es
www.domain.com/it
etc. What is SEO-wise best practice for implementing international index pages. I see following options: entering www.domain.com will display without redirection the index page in language of user (e.g based on IP or browser) in www.domain.com
Example: www.booking.com entering www.domain.com will always show English index page.
Additionally one may display a message in the header if IP from other country with link to other language version.
Example: www.apple.com entering www.domain.com will always redirect automatically to country specific subdirectory based on IP
Example: www.samsung.com Any thoughts/suggestions on what may be best solution from a SEO perspective? For a user I believe options 1) & 3) are preferable.0 -
Why has there been Massive increase in traffic to my clients .eu site after redirects were initiated?
Hi guys, This is a strange one thats really bugging me. I have a client that redirected their domain to a brand new domain that was already live for the previous two months. I have been trying analyse the data however I can't quite understand why there is a massive increase in visitors from the United States when the old site was redirected. The redirection took place at the beginning of July. It was badly managed in terms of the mapping of 301 redirects however thats not the issue here. The level of traffic is gradually decreasing I imagine due to the high level of bounces. The site in question is an EU funded website for education. The old site in the first 2 weeks of June received around 500 visits from the USA while the new site in the first 2 weeks of July (2 weeks into the redirects) received around 3,000 visits from the USA. The new site had previously received only 300 visits for the same period as the old site in the 1st 2 weeks of June. Any idea why this might be? Thanks Rob
International SEO | | daracreative0 -
How to optimise you site in other countries eg Australia
We would like to rank better for specific keywords in Australia. We rank pretty well in our home tld .co.uk but would like to do so in .com.au I would appreciate your thoughts and recommendations.
International SEO | | seanmccauley0