How to prevent duplicate content within this complex website?
-
I have a complex SEO issue I've been wrestling with and I'd appreciate your views on this very much. I have a sports website and most visitors are looking for the games that are played in the current week (I've studied this - it's true). We're creating a new website from scratch and I want to do this is as best as possible. We want to use the most elegant and best way to do this. We do not want to use work-arounds such as iframes, hiding text using AJAX etc. We need a solid solution for both users and search engines.
Therefor I have written down three options:
- Using a canonical URL;
- Using 301-redirects;
- Using 302-redirects.
Introduction
The page 'website.com/competition/season/week-8' shows the soccer games that are played in game week 8 of the season. The next week users are interested in the games that are played in that week (game week 9). So the content a visitor is interested in, is constantly shifting because of the way competitions and tournaments are organized. After a season the same goes for the season of course.
The website we're building has the following structure:
- Competition (e.g. 'premier league')
- Season (e.g. '2011-2012')
- Playweek (e.g. 'week 8')
- Game (e.g. 'Manchester United - Arsenal')
- Playweek (e.g. 'week 8')
- Season (e.g. '2011-2012')
This is the most logical structure one can think of. This is what users expect.
Now we're facing the following challenge: when a user goes to http://website.com/premier-league he expects to see a) the games that are played in the current week and b) the current standings. When someone goes to http://website.com/premier-league/2011-2012/ he expects to see the same: the games that are played in the current week and the current standings. When someone goes to http://website.com/premier-league/2011-2012/week-8/ he expects to the same: the games that are played in the current week and the current standings.
So essentially there's three places, within every active season within a competition, within the website where logically the same information has to be shown.
To deal with this from a UX and SEO perspective, we have the following options:
Option A - Use a canonical URL
Using a canonical URL could solve this problem. You could use a canonical URL from the current week page and the Season page to the competition page:
So:
- the page on 'website.com/$competition/$season/playweek-8' would have a canonical tag that points to 'website.com/$competition/'
- the page on 'website.com/$competition/$season/' would have a canonical tag that points to 'website.com/$competition/'
The next week however, you want to have the canonical tag on 'website.com/$competition/$season/playweek-9' and the canonical tag from 'website.com/$competition/$season/playweek-8' should be removed.
So then you have:
- the page on 'website.com/$competition/$season/playweek-9' would have a canonical tag that points to 'website.com/$competition/'
- the page on 'website.com/$competition/$season/' would still have a canonical tag that points to 'website.com/$competition/'
In essence the canonical tag is constantly traveling through the pages.
Advantages:
- UX: for a user this is a very neat solution. Wherever a user goes, he sees the information he expects. So that's all good.
- SEO: the search engines get very clear guidelines as to how the website functions and we prevent duplicate content.
Disavantages:
- I have some concerns regarding the weekly changing canonical tag from a SEO perspective. Every week, within every competition the canonical tags are updated. How often do Search Engines update their index for canonical tags? I mean, say it takes a Search Engine a week to visit a page, crawl a page and process a canonical tag correctly, then the Search Engines will be a week behind on figuring out the actual structure of the hierarchy. On top of that: what do the changing canonical URLs to the 'quality' of the website? In theory this should be working all but I have some reservations on this.
- If there is a canonical tag from 'website.com/$competition/$season/week-8', what does this do to the indexation and ranking of it's subpages (the actual match pages)
Option B - Using 301-redirects
Using 301-redirects essentially the user and the Search Engine are treated the same. When the Season page or competition page are requested both are redirected to game week page.
The same applies here as applies for the canonical URL: every week there are changes in the redirects.
So in game week 8:
- the page on 'website.com/$competition/' would have a 301-redirect that points to 'website.com/$competition/$season/week-8'
- the page on 'website.com/$competition/$season' would have a 301-redirect that points to 'website.com/$competition/$season/week-8'
A week goes by, so then you have:
- the page on 'website.com/$competition/' would have a 301-redirect that points to 'website.com/$competition/$season/week-9'
- the page on 'website.com/$competition/$season' would have a 301-redirect that points to 'website.com/$competition/$season/week-9'
Advantages
- There is no loss of link authority.
Disadvantages
- Before a playweek starts the playweek in question can be indexed. However, in the current playweek the playweek page 301-redirects to the competition page. After that week the page's 301-redirect is removed again and it's indexable.
- What do all the (changing) 301-redirects do to the overall quality of the website for Search Engines (and users)?
Option C - Using 302-redirects
Most SEO's will refrain from using 302-redirects. However, 302-redirect can be put to good use: for serving a temporary redirect.
Within my website there's the content that's most important to the users (and therefor search engines) is constantly moving. In most cases after a week a different piece of the website is most interesting for a user. So let's take our example above. We're in playweek 8.
If you want 'website.com/$competition/' to be redirecting to 'website.com/$competition/$season/week-8/' you can use a 302-redirect. Because the redirect is temporary
The next week the 302-redirect on 'website.com/$competition/' will be adjusted. It'll be pointing to 'website.com/$competition/$season/week-9'.
Advantages
- We're putting the 302-redirect to its actual use.
- The pages that 302-redirect (for instance 'website.com/$competition' and 'website.com/$competition/$season') will remain indexed.
Disadvantages
- Not quite sure how Google will handle this, they're not very clear on how they exactly handle a 302-redirect and in which cases a 302-redirect might be useful. In most cases they advise webmasters not to use it.
I'd very much like your opinion on this. Thanks in advance guys and galls!
-
Hi Andy and Peter, thanks for your response.
@Andy: the rel=next and rel=prev markup won't really help in solving the problem we had. We will use it though because it's very helpful.
@Peter: yeah it's been something we've been struggling with for a while but we've finally made a decision on it.
The /current solution wasn't really a good solution because at the start of a season all the gameweeks are planned and created so it would become quite complex. We've done some calculations on how much duplicate content we would have if we would not use any of the redirects of canonical tags and the percentage of DC is very small (below 1%) so we're going to put our faith in Google's hands and let them figure it out. It's a good quality website with loads of links we're talking about so I don't expect to much issues. We'll monitor it closely though and stand by to interfere when needed.
Anyways, thanks for your suggestions. Although it didn't solve my problem 1:1 it did make me think and make a decision.
Bye, Steven
-
Yeah, time-sensitive information is always tough. I think you're dead on about the disadvantages - the timing of Google's application of these rotating tags would always be off, and you could end up with some really weird search results that are not only bad for SEO but could create bad UX (people landing on old pages thinking they're new).
What about another option - could you take more of a news/blog approach and have a "/current" page that is always the current week? As the current week changes, roll that content into an archive page ("/week8", etc.). That way, the content lives on, but the current URL never changes.
In terms of duplication, is this really full duplication? It sounds like some pages (like the season) just have snippets of the current week. That's not necessarily a problem. If they are very similar, could you "widgetize" it somehow? Could be straight HTML, but use a condensed format for the season page that links to the full version on the current week page. This would be much like a snippet of a blog post - instead of repeating everything on all 3 pages, have one main chunk of content and two summaries.
-
Hi,
Does the rel=next, rel=prev markup help you out with this problem? See http://googlewebmastercentral.blogspot.co.uk/2011/09/pagination-with-relnext-and-relprev.html
Ive used it a couple of times to help stop pages been seen as dupe content where those pages are duplicate (meta, main content, images etc) except for example reviews, or comments e.g. /product_x /product_x_review_page1 /product_x_review_page2 /product_x_review_page_3
Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
301 redirect to avoid duplicate content penalty
I have two websites with identical content. Haya and ethnic Both websites have similar products. I would like to get rid of ethniccode I have already started to de-index ethniccode. My question is, Will I get any SEO benefit or Will it be harmful if I 301 direct the below only URL’s https://www.ethniccode/salwar-kameez -> https://www.hayacreations/collections/salwar-kameez https://www.ethniccode/salwar-kameez/anarkali-suits - > https://www.hayacreations/collections/anarkali-suits
Intermediate & Advanced SEO | | riyaaaz0 -
Search Causing Duplicate Content
I use Opencart and have found that a lot of my duplicate content (mainly from Products) which is caused by the Search function. Is there a simple way to tell Google to ignore the Search function pathway? Or is this particular action not recommended? Here are two examples: http://thespacecollective.com/index.php?route=product/search&tag=cloth http://thespacecollective.com/index.php?route=product/search
Intermediate & Advanced SEO | | moon-boots0 -
Pages with Duplicate Page Content (with and without www)
How can we resolve pages with duplicate page content? With and without www?
Intermediate & Advanced SEO | | directiq
Thanks in advance.0 -
How should I manage duplicate content caused by a guided navigation for my e-commerce site?
I am working with a company which uses Endeca to power the guided navigation for our e-commerce site. I am concerned that the duplicate content generated by having the same products served under numerous refinement levels is damaging the sites ability to rank well, and was hoping the Moz community could help me understand how much of an impact this type of duplicate content could be having. I also would love to know if there are any best practices for how to manage this type of navigation. Should I nofollow all of the URLs which have more than 1 refinement used on a category, or should I allow the search engines to go deeper than that to preserve the long tail? Any help would be appreciated. Thank you.
Intermediate & Advanced SEO | | FireMountainGems0 -
Robots.txt & Duplicate Content
In reviewing my crawl results I have 5666 pages of duplicate content. I believe this is because many of the indexed pages are just different ways to get to the same content. There is one primary culprit. It's a series of URL's related to CatalogSearch - for example; http://www.careerbags.com/catalogsearch/result/index/?q=Mobile I have 10074 of those links indexed according to my MOZ crawl. Of those 5349 are tagged as duplicate content. Another 4725 are not. Here are some additional sample links: http://www.careerbags.com/catalogsearch/result/index/?dir=desc&order=relevance&p=2&q=Amy
Intermediate & Advanced SEO | | Careerbags
http://www.careerbags.com/catalogsearch/result/index/?color=28&q=bellemonde
http://www.careerbags.com/catalogsearch/result/index/?cat=9&color=241&dir=asc&order=relevance&q=baggallini All of these links are just different ways of searching through our product catalog. My question is should we disallow - catalogsearch via the robots file? Are these links doing more harm than good?0 -
Ecommerce Duplicate Product Descriptions across 3 websites
Hi, We are an e commerce company that has our own domain but also sell the same products on eBay and Amazon. What is the feeling on the same exact descriptions being used on different platforms? Do they count as duplicate content? Will our domain be punished/penalised as our domain does not have as much authority as EBay or Amazon? We have over 5,000 products with our own hand written product descriptions. We want our website to be the main place/ have priority over the above market places. What's the best suggestion/solution? thanks,
Intermediate & Advanced SEO | | Roy19730 -
Proper Hosting Setup to Avoid Subfolders & Duplicate Content
I've noticed with hosting multiple websites on a single account you end up having your main site in the root public_html folder, but when you create subfolders for new website it actually creates a duplicate website: eg. http://kohnmeat.com/ is being hosted on laubeau.com's server. So you end up with a duplicate website: http://laubeau.com/kohn/ Anyone know the best way to prevent this from happening? (i.e. canonical? 301? robots.txt?) Also, maybe a specific 'how-to' if you're feeling generous 🙂
Intermediate & Advanced SEO | | ATMOSMarketing560 -
PDF on financial site that duplicates ~50% of site content
I have a financial advisor client who has a downloadable PDF on his site that contains about 9 pages of good info. Problem is much of the content can also be found on individual pages of his site. Is it best to noindex/follow the pdf? It would be great to let the few pages of original content be crawlable, but I'm concerned about the duplicate content aspect. Thanks --
Intermediate & Advanced SEO | | 540SEO0