Hreflang tags and canonical tags - might be causing indexing and duplicate content issues
-
Hi,
Let's say I have a site located at https://www.example.com, and also have subdirectories setup for different languages. For example:
https://www.example.com/es_ES/
https://www.example.com/fr_FR/
https://www.example.com/it_IT/
My Spanish version currently has the following hreflang tags and canonical tag implemented:
My robots.txt file is blocking all of my language subdirectories. For example:
User-agent:*
Disallow: /es_ES/
Disallow: /fr_FR/
Disallow: /it_IT/
This setup doesn't seem right. I don't think I should be blocking the language-specific subdirectories via robots.txt
What are your thoughts?
Does my hreflang tag and canonical tag implementation look correct to you? Should I be doing this differently?
I would greatly appreciate your feedback and/or suggestions.
-
Hi... I'm sorry to tell you that the answer offered by Gaston is not totally correct.
So, in your Spanish page you have these hreflang and canonical annotations:
This is not correct because you are not adding also the self-referential hreflang annotation
Google is very precise about this, and it states its need in the help pages as well in many Googlers tweets and webmaster office hangouts.
The rel="canonical" is correct. Remember that the self-referential and the alternative href URLs must always be canonicals.
Finally, regarding the subfolders blocked via robots.txt, yes! that's totally incorrect:
if you're blocking Googlebot from accessing the Spanish, French and Italian subfolders, then Googlebot won't be able to parse the code of their pages, hence it won't be able to see also the hreflang annotations... with obvious erroneous consequences.
-
that's corect.
-
Yes. example.com/en-US/ redirects 301 to example.com
So, when referencing that version in hreflang will it look like this?
Is this correct?
-
So, every page of example.com/en-US/ redirects 301 to example.com/ ?
If that's the case, then there is no reason in having that folder (/en-US/), just when configuring Hreflang for en-US use the URL without that folder
-
What do you mean by "exactly the same then there should not be 2 sites"?
My https://www.example.com/en-US/ 301 redirects to https://www.example.com
Thoughts?
-
Yeap, those implementations are correct.
in order to avoid duplicate content between different language/countries websites, in each page that is present on each site, there should be their corresponding hreflang tag.
In your case:
And for a sample page: https://www.smarttechMKT.com/es-ES/gastonriera-espanolNOTE, in the case that site.com and site.com/en-us/ are exactly the same then there should not be 2 sites. Just the one without the folder and hreflang tag with en-US should point to that
Hope it helps.
Best luck.
GR -
Thank you for the response, Gaston! I really appreciate it.
So you are certain that my hreflang and canonical tags are implemented correctly? For example, my Spanish version (https://www.example.com/es_ES/
Is this implementation correct?
Also, will I have any duplicate content issues with these different language versions?
-
Hello there,
Watch out your question, there is a site there. If its your clients, edit it.
Canonical and hreflang seems OK.
Blocking other languages/countries is wrong. There you are allowing google to see ONLY the us version.For further reading, take a look at these articles:
Hreflang:
Multi-regional and multilingual sites - Google Search Console
International checklist - Moz Blog
Using the correct hreglang tag - Moz Blog
Guide to international website expansion - Moz Blog
Tool for checking hreflang anotations - Moz Blog Canonicals:
SEO Best Practices for Canonical URLs + the Rel=Canonical Tag - Whiteboard Friday Consolidate duplicate URLs - Google Search Console HelpHope it helps.
Best Luck.
GR.Hope it helps.
Best luck.
GR.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
For My International Sites only Homepage in other Language rest Pages are in English. Hreflang required here?
Hello All, For my ecommerce site at my homepage there is an Language option of 9 different countries. My main site - abcd.co.uk and other sites are like this se.abcd.co.uk, fr.abcd.co.uk, es.abcd.co.uk etc From my main site if user clicks on fr.abcd.co.uk then France site will open but when he click on any link it will redirect to my UK site. On France site homepage if user hover the cursor then links are visible of UK site only. My query is ;- Do it required here to implement hreflang? As only homepage is in different language? Do it anything wrong in google point of view? Thanks!
International SEO | | wright3350 -
If I redirect based on IP will Google still crawl my international sites if I implement Hreflang
We are setting up several international sites. Ideally, we wouldn't set up any redirects, but if we have to (for merchandising reasons etc) I'd like to assess what the next best option would be. A secondary option could be that we implement the redirects based on IP. However, Google then wouldn't be able to access the content for all the international sites (we're setting up 6 in total) and would only index the .com site. I'm wondering whether the Hreflang annotations would still allow Google to find the International sites? If not, that's a lot of content we are not fully benefiting from. Another option could be that we treat the Googlebot user agent differently, but this would probably be considered as cloaking by the G-Man. If there are any other options, please let me know.
International SEO | | Ben.JD0 -
Multi-Country Duplicate Content
Hello, We have an ecommerce site that serves several countries on the same .com domain - US, UK and CA. We have duplicate content across these countries because they are all English speaking so there is little variance in the pages and they each sell most of the same products. We have implemented hreflang into our sitemaps but we need to address the duplicate content. We were advised to canonicalize our UK and CA pages back to the duplicate US pages (our US pages account for the majority of our traffic and sales). This would cause the UK and CA pages to fall out of the index but the visitor would still be taken to the correct country's page due to the hreflang. I'm leary about doing this because they are across countries. Is this ok to do? If not, how do we address the duplicate content since they are not on their own CCTLD's?
International SEO | | Colbys0 -
Delivering different content according to country
Hey, I have a question regarding different content according to country (IP)-
International SEO | | Kung_fu_Panda
We planing to serve mobile users using dynamic HTML serving (on the same url)
Is it possible to serve different content for different devices + different IPs (for example different content for a user from US android and someone from UK android ) thanks!0 -
Geotarget subfolders with the same language or get rid of duplicates but lose option to geotarget?
Hi, we have a domain that is aimed to cover LatAm region. Currently, the homepage contains country selector for ~20 countries. 95% of them hold content in Spanish. We have only homepages for each regions as separate subfolders, i.e.
International SEO | | eset
www.maindomain.com/co
www.maindomain.com/cl
www.maindomain.com/br
etc. but once the user clicks on menu item he is taken back to main domain subpages, i.e. www.maindomain.com/comprar My struggle is to decide whether it is better to: A) copy all content for each subfolder, which will create huge amount of duplicates (there are no resources to create unique content and it is even impossible taking into account nature of the product - mostly tech.specs, etc.) and implement hreflang sitemaps and configure GWT to target each country with its own Spanish content (the same for each country) OR B) remove all local subfolders and keep only main domain in Spanish that will serve all countries within the region. With this option, we will get rid of duplicates but also lose option to geotarget. So, my questions is which option will do less harm, or if there is any other approach that comes to your minds. I consulted with two agencies but still haven't got clear answer. Thanks a lot for your help!0 -
How to fix the duplicate content problem on different domains (.nl /.be) of your brand's websites in multiple countries?
Dear all, what is the best way to fix the duplicate content problem on different domains (.nl /.be) of your brand's websites in multiple countries? What must I add to my code of websites my .nl domain to avoid duplicate content and to keep the .nl website out of google.be, but still well-indexed in google.nl? What must I add to my code of websites my .be domain to avoid duplicate content and to keep the .nl website out of google.be, but still well-indexed in google.nl? Thanks in advance!
International SEO | | HMK-NL3 -
Non US site pages indexed in US Google search
Hi, We are having a global site wide issue with non US site pages being indexed by Google and served up in US search results. Conversley, we have US en pages showing in the Japan Google search results. We currently us IP detect to direct users to the correct regional site but it isn't effective if the users are entering through an incorrect regional page. At the top of each or our pages we have a drop down menu to allow users to manually select their preferred region. Is it possible that Google Bot is crawling these links and indexing these other regional pages as US and not detecting it due to our URL structure? Below are examples of two of our URLs for reference - one from Canada, the other from the US /ca/en/prod4130078/2500058/catalog50008/ /us/en/prod4130078/2500058/catalog20038/ If that is, in fact, what is happening, would setting the links within the drop down to 'no follow' address the problem? Thank you. Angie
International SEO | | Corel0 -
Google Indexing Part two
Hi Everybody, I am trying to understand how does Google works, so I ve been reading and researching a lot. But I am still having a problem that I cannot solve. My website is in several languages, but its main language is Catalan. so if you get into my webite: "www.vallnord.com" the default language will be catalan. but if someone using Google.es in Spanish I would like the spanish version of the web to be the main result not the catalan. unfortunately this does not work like this. For a search query like "Esqui Andorra" the catalan version is on the 1st page and the spanish (www.vallnord.com/es) is on the 4th page. Does anybody know why is this happening or how can I solve it? Regards.
International SEO | | SilbertAd0