Do you think the SEs would see this as duplicate content?
-
Hi Mozzers!
I have a U.S. website and a Chinese version of that U.S. website.
The China site only gets direct and PPC traffic because the robots.txt file is disallowing the SEs from crawling it.
Question: If I added English sku descriptions and English content to the China site (which is also on our U.S. site), will the SEs penalize us for duplicate content even though the robots.txt file doesn’t allow them to see it?
I plan on translating the descriptions and content to Chinese at a later date, but wanted to ask if the above was an issue.
Thanks Mozzers!
-
Your robot text should play no part in this. You should leave the robots.txt however it should normally be for the website. Google knows that if you're serving a different country with a different IP along with a TLD that you will not be infringing on their rules regarding duplicate content because it is natural for somebody to have one site in one country and another site in another country and have the exact same content on those sites but therefore different target audiences so they're not gonna come up in the Google search rankings and they will both be good results for each country's audience.
Do not block anything with robots.txt that you do not need to block otherwise
Long story short if you're using robots.txt to block anything do not worry about that you can remove that block
-
Hi Thomas. Thanks again.
We have separate domains in separate countries--I think we're set there.
It is the question of having dupe content or not on the sites when one site has robots.txt turned "off".
-
To see the best practices on where to host for individual countries check out this whiteboard Friday
http://moz.com/blog/international-seo-where-to-host-and-how-to-target-whiteboard-friday
Have a answered all your questions?
All the best,
Thomas
-
Happy to be of help. It definitely will help you to have the domain hosted by a host inside whatever country you are targeting.
I'm glad to be of help sincerely,
Thomas
-
Thanks Thomas!
I should point out that the U.S. domain is hosted in the U.S. and the China domain is hosted in China.
Not sure if that makes a lick of difference.
-
If you have a Chinese version and a US version and the end with the different TDL you will have no penalties from Google you can keep the exact same content though you should obviously have the translation in place already.
You can do this without any worry of duplicate content whatsoever
For example I could have two sites example.co.uk and example.com and have them have identical content however I will not be penalized by Google whatsoever even those words are completely the same even in the same language because there for different countries.
Sincerely,
Thomas
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hreflang tags and canonical tags - might be causing indexing and duplicate content issues
Hi, Let's say I have a site located at https://www.example.com, and also have subdirectories setup for different languages. For example: https://www.example.com/es_ES/ https://www.example.com/fr_FR/ https://www.example.com/it_IT/ My Spanish version currently has the following hreflang tags and canonical tag implemented: My robots.txt file is blocking all of my language subdirectories. For example: User-agent:* Disallow: /es_ES/ Disallow: /fr_FR/ Disallow: /it_IT/ This setup doesn't seem right. I don't think I should be blocking the language-specific subdirectories via robots.txt What are your thoughts? Does my hreflang tag and canonical tag implementation look correct to you? Should I be doing this differently? I would greatly appreciate your feedback and/or suggestions.
International SEO | | Avid_Demand0 -
How to deal with disproportional content investment for a ccTLD for a multi-language country,
We have a website for the Belgium market, serving content and products on be/nl (Dutch/Flemmish Belgium) and .be/fr (French Belgium). However, as a Dutch-based company you can see our primary focus and objective is to serve content to Dutch Belgium rather than French Belgium. I wonder if, and so, what are the downsides are of only investing in half of the site?
International SEO | | Marketing-Omoda
Does it hurt my general .be Google rankings if we put a lot of effort in .be/nl but far less in .be/fr ? (we used to have a ccTLD .fr as well, but pulled the plug because it wasn't profitable.
our belgium website is profitable for Dutch speaking part of Belgium but now we would like to expand, and enhance rankings. We're investing heavily in (local) brand awareness and partnerships, and content marketing for the Dutch part.0 -
International SEO & Duplicate Content: ccTLD, hreflang, and relcanonical tags
Hi Everyone, I have a client that has two sites (example.com & example.co.uk) each have the same English content, but no hreflang or rel="canonical" tags in place. Would this be interpreted as duplicate content? They haven't changed the copy to speak to specific regions, but have tried targeting the UK with a ccTLD. I've taken a look at some other comparable question on MOZ like this post - > https://moz.com/community/q/international-hreflang-will-this-handle-duplicate-content where one of the answers says **"If no translation is happening within a geo-targeted site, HREFLANG is not necessary." **If hreflang tags are not necessary, then would I need rel="canonical" to avoid duplicate content? Thanks for taking the time to help a fellow SEO out.
International SEO | | ccox10 -
International Sites and Duplicate Content
Hello, I am working on a project where have some doubts regarding the structure of international sites and multi languages.Website is in the fashion industry. I think is a common problem for this industry. Website is translated in 5 languages and sell in 21 countries. As you can imagine this create a huge number of urls, so much that with ScreamingFrog I cant even complete the crawling. Perhaps the UK site is visible in all those versions http://www.MyDomain.com/en/GB/ http://www.MyDomain.com/it/GB/ http://www.MyDomain.com/fr/GB/ http://www.MyDomain.com/de/GB/ http://www.MyDomain.com/es/GB/ Obviously for SEO only the first version is important One other example, the French site is available in 5 languages and again... http://www.MyDomain.com/fr/FR/ http://www.MyDomain.com/en/FR/ http://www.MyDomain.com/it/FR/ http://www.MyDomain.com/de/FR/ http://www.MyDomain.com/es/FR/ And so on...this is creating 3 issues mainly: Endless crawling - with crawlers not focusing on most important pages Duplication of content Wrong GEO urls ranking in Google I have already implemented href lang but didn't noticed any improvements. Therefore my question is Should I exclude with "robots.txt" and "no index" the non appropriate targeting? Perhaps for UK leave crawable just English version i.e. http://www.MyDomain.com/en/GB/, for France just the French version http://www.MyDomain.com/fr/FR/ and so on What I would like to get doing this is to have the crawlers more focused on the important SEO pages, avoid content duplication and wrong urls rankings on local Google Please comment
International SEO | | guidoampollini0 -
Do I have duplicate content issues to be worried about?
Hey guys, We built a website http://www.cylon.com/ targeting different regions but with the same English langauage (Ireland, England and America). The content for the most part is the same set up on 3 different subfolders. http://www.cylon.com/ - Targeting United States in WMT http://www.cylon.com/ie - Targeting Ireland in WMT http://www.cylon.com/uk - Targeting UK in WMT Do I have duplicate content issues to be worried about? If so, how do I get around this issue? Also is there anyway of finding out if Google have in some way penalised these pages for having the same content on other pages trageting different Countries? I have not received any messages from Google in WMT saying there is duplicate so I'm not sure if this is an issue. Thanks Rob
International SEO | | daracreative0 -
Delivering different content according to country
Hey, I have a question regarding different content according to country (IP)-
International SEO | | Kung_fu_Panda
We planing to serve mobile users using dynamic HTML serving (on the same url)
Is it possible to serve different content for different devices + different IPs (for example different content for a user from US android and someone from UK android ) thanks!0 -
Will duplicate content across international domains have a negative affect on our SERP
Our corporate website www.tryten.com showcases/sells our products to all of our customers. We have Canadian and UK based customers and would like to duplicate our website onto .ca and .co.uk domains respectively to better service them. These sites will showcase the same products, only the price and ship from locations will change. Also, the phone numbers and contact info will be altered. The sites will all be on one server. On each of the sites there will be a country selector which will take you to the appropriate domain for the country selected. Will doing this negatively affect our rankings in the US, UK and Canada?
International SEO | | tryten0 -
Differents TLDs and same contents not a problem Matt Cutts says?
Matt Cutts says on this video that you can have the same content on different TLDs and there is no duplicate content for Google. Have someone try this experience? For example : same content on "mysite.fr" and "mysite.be". And for the visitors from Belgium, will they see into the SERPs "mysite.be" and for the visitors from France "mysite.fr"? Thank you for your answer guys. Jon watch?v=Ets7nHOV1Yo&feature=player_embedded
International SEO | | JonathanLeplang1