Do you think the SEs would see this as duplicate content?
-
Hi Mozzers!
I have a U.S. website and a Chinese version of that U.S. website.
The China site only gets direct and PPC traffic because the robots.txt file is disallowing the SEs from crawling it.
Question: If I added English sku descriptions and English content to the China site (which is also on our U.S. site), will the SEs penalize us for duplicate content even though the robots.txt file doesn’t allow them to see it?
I plan on translating the descriptions and content to Chinese at a later date, but wanted to ask if the above was an issue.
Thanks Mozzers!
-
Your robot text should play no part in this. You should leave the robots.txt however it should normally be for the website. Google knows that if you're serving a different country with a different IP along with a TLD that you will not be infringing on their rules regarding duplicate content because it is natural for somebody to have one site in one country and another site in another country and have the exact same content on those sites but therefore different target audiences so they're not gonna come up in the Google search rankings and they will both be good results for each country's audience.
Do not block anything with robots.txt that you do not need to block otherwise
Long story short if you're using robots.txt to block anything do not worry about that you can remove that block
-
Hi Thomas. Thanks again.
We have separate domains in separate countries--I think we're set there.
It is the question of having dupe content or not on the sites when one site has robots.txt turned "off".
-
To see the best practices on where to host for individual countries check out this whiteboard Friday
http://moz.com/blog/international-seo-where-to-host-and-how-to-target-whiteboard-friday
Have a answered all your questions?
All the best,
Thomas
-
Happy to be of help. It definitely will help you to have the domain hosted by a host inside whatever country you are targeting.
I'm glad to be of help sincerely,
Thomas
-
Thanks Thomas!
I should point out that the U.S. domain is hosted in the U.S. and the China domain is hosted in China.
Not sure if that makes a lick of difference.
-
If you have a Chinese version and a US version and the end with the different TDL you will have no penalties from Google you can keep the exact same content though you should obviously have the translation in place already.
You can do this without any worry of duplicate content whatsoever
For example I could have two sites example.co.uk and example.com and have them have identical content however I will not be penalized by Google whatsoever even those words are completely the same even in the same language because there for different countries.
Sincerely,
Thomas
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hreflang tags and canonical tags - might be causing indexing and duplicate content issues
Hi, Let's say I have a site located at https://www.example.com, and also have subdirectories setup for different languages. For example: https://www.example.com/es_ES/ https://www.example.com/fr_FR/ https://www.example.com/it_IT/ My Spanish version currently has the following hreflang tags and canonical tag implemented: My robots.txt file is blocking all of my language subdirectories. For example: User-agent:* Disallow: /es_ES/ Disallow: /fr_FR/ Disallow: /it_IT/ This setup doesn't seem right. I don't think I should be blocking the language-specific subdirectories via robots.txt What are your thoughts? Does my hreflang tag and canonical tag implementation look correct to you? Should I be doing this differently? I would greatly appreciate your feedback and/or suggestions.
International SEO | | Avid_Demand0 -
Duplicate content - news archive
Most of them are due to news items having more than 1 category – which is pretty normal.Also /us/blog, /uk/blog and /ca/blog are effectively the same page.None of them are actually duplicate content – just alternate URLs for the same pagehttp://www.fdmgroup.com/category/news/
International SEO | | fdmgroup0 -
What is the current thinking about translated versions of pages? Is it necessary
My company is about to do a big push in China. We can get our homepage translated in Chinese at a very reasonable price. My questions are: Is it worth it? Do browsers to an adequately job of translating pages? If it is worth it: Can someone suggest a good post explaining what to do with the translation? What are the SEO implications? Thank you
International SEO | | appbackr
Sarah0 -
Do I have duplicate content issues to be worried about?
Hey guys, We built a website http://www.cylon.com/ targeting different regions but with the same English langauage (Ireland, England and America). The content for the most part is the same set up on 3 different subfolders. http://www.cylon.com/ - Targeting United States in WMT http://www.cylon.com/ie - Targeting Ireland in WMT http://www.cylon.com/uk - Targeting UK in WMT Do I have duplicate content issues to be worried about? If so, how do I get around this issue? Also is there anyway of finding out if Google have in some way penalised these pages for having the same content on other pages trageting different Countries? I have not received any messages from Google in WMT saying there is duplicate so I'm not sure if this is an issue. Thanks Rob
International SEO | | daracreative0 -
Should I be deindexing pages with thin or weak content?
If I have pages that rank product categories by alphabetical order should I deindex those pages? Keeping in mind the pages do not have any content apart from product titles? For example: www.url.com/albums/a/ www.url.com/albums/b/ If I deindexed these pages would I lose any authority passed through internal linking?
International SEO | | Jonathan_Hatton0 -
E-Commerce site in 2 languages - Duplicate content or not?
How does Google view this? Our current site works like:
International SEO | | bjs2010
www.domain.com/EN - English
www.domain.com/ES - Spanish All products are the same, just different language and different URL for them - is this good or bad? I thought of either Going with .co.uk or .com for "English" and a .es for "Spanish"
OR Subdomaining it. www.es.domain.com and www.en.domain.com Any advice appreciated!0 -
Duplicated 404 Pages (Travel Industry)
Our website has creating numberous "future pages" with no alt tag or class tag that are showing up as 404 pages, To make matters worst, they are causing duplicate 404 pages because we have different languages. The visitors cant find the 404s but the searchbots can. Would it better to remove or add the links to robot.txt or add nofollow/noindex tag? This is an example. http://www.solmelia.com/nGeneral.LINK_FAQ http://www.solmelia.com/nGeneral.LINK_HOTELESDESTINOS_BODAS http://www.solmelia.com/nGeneral.LINK_CONDICIONES http://www.solmelia.com/nGeneral.LINK_MAPSITE http://www.solmelia.com/nGeneral.LINK_HOTELESDESTINOS_EMPRESA
International SEO | | Melia0 -
Internationally targetted subdomains and Duplicate content
A client has a site they'd like to translated into French, not for the french market but for french speaking countries. My research tells me the best way to implement this for this particular client is to create subfolders for each country. For ease of implementation I’ve decided against ccTLD’s and Sub Domains. So for example… I'll create www.website.com/mr/ for Mauritania and in GWT set this to target Mauritania. Excellent so far. But then I need to build another sub folder for Morocco. I'll then create www.website.com/ma/ for Morocco and in GWT set this to target Morocco. Now the content on these two sub folders will be exactly the same and I’m thinking about doing this for all French speaking African countries. It would be nice to use www.website.com/fr/ but in GWT you can only set one Target country. Duplicate content issues arise and my fear of perturbing the almighty Google becomes a possibility. My research indicates that I should simply canonical back to the page I want indexed. But I want them both to be indexed surely!? I therefore decided to share my situation with my fellow SEO’s to see if I’m being stupid or missing something simple both a distinct possibility!
International SEO | | eazytiger0