Multiple Countries, Same Language: Receiving Duplicate Page & Content Errors
-
Hello!
I have a site that serves three English-speaking countries, and is using subfolders for each country version:
- United Kingdom: https://site.com/uk/
- Canada: https://site.com/ca/
- United States & other English-speaking countries: https://site.com/en/
The site displayed is dependent on where the user is located, and users can also change the country version by using a drop-down flag navigation element in the navigation bar. If a user switches versions using the flag, the first URL of the new language version includes a language parameter in the URL, like:
In the Moz crawl diagnostics report, this site is getting dinged for lots of duplicate content because the crawler is finding both versions of each country's site, with and without the language parameter.
However, the site has rel="canonical" tags set up on both URL versions and none of the URLs containing the "?language=" parameter are getting indexed.
So...my questions:
1. Are the Duplicate Title and Content errors found by the Moz crawl diagnostic really an issue?
2. If they are, how can I best clean this up?
Additional notes: the site currently has no sitemaps (XML or HTML), and is not yet using the hreflang tag. I intend to create sitemaps for each country version, like:
- .com/en/sitemap.xml
- .com/ca/sitemap.xml
- .com/uk/sitemap.xml
I thought about putting a 'nofollow' tag on the flag navigation element, but since no sitemaps are in place I didn't want to accidentally cut off crawler access to alternate versions.
Thanks for your help!
-
Yep, given your resource constraints, I'd focus on translations for now. If you ever get to a point that there is something bigger than price differentiating your content, then you can think about geo-targeting. You will need the resources to differentiate the content though.
Right now, my recommendation is to drop the country specific content and just offer English for now. Your content can rank for any English speaking search, regardless of country. However, if the terms people use in the US, UK and Canada differ that much, you can "translate" the content (en-us, en-gb, en-au) and use the HREFLANG tag.
For price changes, that's tricker, but do you offer the price in search results via schema? Does it show up? If not, then you can use cookies to set the prices dependent on the country the person chooses (try not to use IP address, and if you do, make people confirm the setting).
For now, focus your time and efforts getting the flow right for the user. Only worry about HREFLANG if your English content needs to be differentiated for term usage. Then focus your efforts on getting those upcoming translations right. When that is ready, then really use HREFLANG.
Hope that helps!
-
Hi Kate,
Nifty quiz and flowchart! Thanks for sharing it. All the countries targeted are English-speaking, though further expansion to non-English speaking countries is planned for 2015. Here are the answers to the questions:
1. Does your business/product/content change in different countries?
A: Not really. 90% of the products are available in all three countries, and only one country is currently lacking the remaining 10%, and it will start selling those products there in 2015.
2. Would it make sense to an international visitor to see different site content? (ex. currency, localization, etc.)
A: Currency - yes. Otherwise, not really.
3. Do you have the resources to differentiate the content?
A: Not currently. This is a set of branded products, and the product descriptions use extensive "on-brand" language.
4. Are there multiple official languages for any of these countries?
A: Yes, Canada's official languages are English and French. There is no French version currently available.
5. Do you plan on offering the site content in all official languages?
A: Next six months - no. Late 2015 - maybe.
Going through the quiz, if I answer:
1. No, 2. Yes, 3. No
This is the recommendation:
Your International Strategy is:
Translate Only
- Don’t machine translate, while manual translation is costly, it’s the best for your goals.
- Put your HREFLANG in XML sitemaps.
- Use the Language Meta tag for Bing translation targeting.
- Don’t use a ccTLD. That is for Geo-Targeting only.
Aside from the manual translation portion, do you think #2 and #3 are still the best solutions for this situation?
Thanks for your help!
-
Hi!
This is a tough one because I can't tell if you mean to be geo-targeting or translate. It's not a one or the other thing, but it usually is when you are just targeting english speaking countries. Can you do me a favor and go to http://www.katemorris.com/issg/ and go through the questions there? Let me know what the "answer" is for your situation and I'll help you get to the right solution.
But in short, yes, the duplicate content is a real issue with or without the lang parameter.
Let me know!
-
Oh this is a tough one. The problem is that no matter the tags and language, the content is the same. It is reflecting duplicate content because it is duplicate content. Duplicate content within your site is serious, especially if you are trying to target keywords on those pages.
The hreflang tags should help you be able to display languages without using so many duplicate pages. I don't have much experience with that tag, but my advice would be to look into it further to help with your duplicate content issue. No following the duplicate pages will ultimately effect their rankings, so that probably isn't the best thing to do.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Large site with content silo's - best practice for deep indexing silo content
Thanks in advance for any advice/links/discussion. This honestly might be a scenario where we need to do some A/B testing. We have a massive (5 Million) content silo that is the basis for our long tail search strategy. Organic search traffic hits our individual "product" pages and we've divided our silo with a parent category & then secondarily with a field (so we can cross link to other content silo's using the same parent/field categorizations). We don't anticipate, nor expect to have top level category pages receive organic traffic - most people are searching for the individual/specific product (long tail). We're not trying to rank or get traffic for searches of all products in "category X" and others are competing and spending a lot in that area (head). The intent/purpose of the site structure/taxonomy is to more easily enable bots/crawlers to get deeper into our content silos. We've built the page for humans, but included link structure/taxonomy to assist crawlers. So here's my question on best practices. How to handle categories with 1,000+ pages/pagination. With our most popular product categories, there might be 100,000's products in one category. My top level hub page for a category looks like www.mysite/categoryA and the page build is showing 50 products and then pagination from 1-1000+. Currently we're using rel=next for pagination and for pages like www.mysite/categoryA?page=6 we make it reference itself as canonical (not the first/top page www.mysite/categoryA). Our goal is deep crawl/indexation of our silo. I use ScreamingFrog and SEOMoz campaign crawl to sample (site takes a week+ to fully crawl) and with each of these tools it "looks" like crawlers have gotten a bit "bogged down" with large categories with tons of pagination. For example rather than crawl multiple categories or fields to get to multiple product pages, some bots will hit all 1,000 (rel=next) pages of a single category. I don't want to waste crawl budget going through 1,000 pages of a single category, versus discovering/crawling more categories. I can't seem to find a consensus as to how to approach the issue. I can't have a page that lists "all" - there's just too much, so we're going to need pagination. I'm not worried about category pagination pages cannibalizing traffic as I don't expect any (should I make pages 2-1,000) noindex and canonically reference the main/first page in the category?). Should I worry about crawlers going deep in pagination among 1 category versus getting to more top level categories? Thanks!
Moz Pro | | DrewProZ1 -
Duplicate content issues with file download links (diff. versions of a downloadable application)
I'm a little unsure how canonicalisation works with this case. 🙂 We have very regular updates to the application which is available as a download on our site. Obviously, with every update the version number of the file being downloaded changes; and along with it, the URL parameter included when people click the 'Download' button on our site. e.g. mysite.com/download/download.php?f=myapp.1.0.1.exe mysite.com/download/download.php?f=myapp.1.0.2.exe mysite.com/download/download.php?f=myapp.1.0.3.exe, etc In the Moz Site Crawl report all of these links are registering as Duplicate Content. There's no content per se on these pages, all they do is trigger a download of the specified file from our servers. Two questions: Are these links actually hurting our ranking/authority/etc? Would adding a canonical tag to the head of mysite.com/download/download.php solve the crawl issues? Would this catch all of the download.php URLs? i.e. Thanks! Jon
Moz Pro | | jonmc
(not super up on php, btw. So if I'm saying something completely bogus here...be kind 😉 )0 -
Crawl Diagnostics saids a page is linking but I can't find the link on the page.
Hi I have just got my first Crawl Diagnostics report and I have a questions. It saids that this page: http://goo.gl/8py9wj links to http://goo.gl/Uc7qKq which is a 404. I can't recognize the URL on the page which is a 404 and when searching in the code I can't find the %7Blink%7D in the URL which gives the problems. I hope you can help me to understand what triggers it 🙂
Moz Pro | | SebastianThode0 -
Are there tools to discover duplicate content issues with the other websites?
We have issues with users copy-pasting content from other sources into our site. The only way I know to find out, is to manually (!!) copy a snippet of their text into google, to see if I get results from other sites. I have been googling for tools to help automate this process, but without luck. Can you recommend any?
Moz Pro | | betternow0 -
On Link Analysis tab I my best pages are 301 and 404 pages.
I looked on my redirrect file and found that /* redirects to /v/404.asp.
Moz Pro | | sbetzen
However if you look below at the link analysis the 404 page is getting a 404 error.
The homepage ecowindchimes.com/ is getting a 301 (but I don't know where it is going to).
The third one is also redirected. 1. [No Data] ecowindchimes.com/ ||| 301 ||| 2 ||| 36 2. 2. [No Data] ecowindchimes.com/v/404.asp ||| 404 ||| 2 ||| 34 3. [No Data] 3. ecowindchimes.com/index.html?lang=en-us&target=d2.html ||| 301 ||| 1 ||| 33 So I have 2 questions: 1) should this be fixed? and 2) how? This is a volusion site and I believe the "catchall" redirect was done by them0 -
Page Rank Report says #6 in Google but I can't find the page anywhere
So SEOMoz says that I've consistently ranked #6 for a certain keyword. But when I search I'm no where to be found. I've done regular searches, incognito and some non-seomoz reports and all come up with nothing in Google. I noticed it a week or two ago, but didn't think it would continue. This is no bueno. I wouldn't be surprised if I got penalized (luckily my homepage relatively well for similar keywords), an old seo consultant used very spammy tactics. I recently removed them, but not before I started to notice that I fell off the map. Why would SEOMoz not recognize this, and continue to say I'm ranking well? The keyword is bpi building analyst the page is http://www.cleanedison.com/courses/bpi-building-analyst
Moz Pro | | CleanEdisonInc0 -
Duplicate Content Issues with WordPress
I'm having some difficulty with a few of the sites I'm managing right now. When I run a report here, I'm getting a duplicate content issue with sites that I'm running through WordPress. Sites running on a different CMS are not getting the issue. The duplicate content is being listed as from two URL's that are identical. I checked trailing slash, spelling, capitalization, everything. It looks like the same site is being marked as two with duplicate content. Does anyone have any ideas of what could be causing this and/or what I may be able to do to resolve the issue (or if it's really something to worry about or not)? Thanks. (and thanks for helping the new guy!)
Moz Pro | | DeliaAssociates0 -
On page optimisation tool issues
When viewing my campaign and looking at the on page optimisation tool, I have a few issues. I seems to only shows the keywords I want rankings for and how optimised my homepage is for those keywords. Is there any way I can get it to analyse permanently specifc keywords for specific pages because my homepage isnt optimised for some keywords which are on my list, which I have optimised other pages for, and because its looking at my homepage its getting a really low grade, and looks really bad and frustrates me because I cant work this out. Any help greatly appreciated.
Moz Pro | | CompleteOffice1