Duplicate Content & Rel Canonical Tag not working
-
I'm really questioning the legitimacy of the duplicate content flags with moz. I'm building a website that sells home decor products and a lot of the pages are similar in structure (As would be expected with a store that sells thousands of individual products). It seems a little overkill to me to flag the following pages as duplicate content. They have different urls, titles, h1, h2, and h3 tages, different meta tags, etc. Right now, it's saying that the following have duplicate page content:
http://www.countryporchhomedecor.com
http://countryporchhomedecor.com/park-designs/pillows/christmas-vacation-embroidered-pillow
http://www.countryporchhomedecor.com/donna-sharp/throws/camo-bear-throw
http://countryporchhomedecor.com/park-designs/teapots/wonderland-teapot
http://countryporchhomedecor.com/park-designs/rag-rugs/cambridge-rug-36x60
http://www.countryporchhomedecor.com/donna-sharp/lodge-quilts/king%2C-woodland
http://www.countryporchhomedecor.com/park-designs/rag-rugs/redmon-rag-rug-36x60
http://www.countryporchhomedecor.com/park-designs/valances/hearthside-valance-72x14
http://countryporchhomedecor.com/park-designs/valances/hearthside-valance-72x14
http://countryporchhomedecor.com/donna-sharp/lodge-quilts/king,-woodland
http://www.countryporchhomedecor.com/park-designs/teapots/wonderland-teapot
http://countryporchhomedecor.com/donna-sharp/throws/camo-bear-throw
http://countryporchhomedecor.com/park-designs/accessories/home-place-tumbler
http://www.countryporchhomedecor.com/donna-sharp/lodge-quilts/king,-woodland
http://www.countryporchhomedecor.com/park-designs/rag-rugs/cambridge-rug-36x60
http://www.countryporchhomedecor.com/park-designs/pillows/christmas-vacation-embroidered-pillow
http://www.countryporchhomedecor.com/donna-sharp/lodge-quilts/king%2C-woodland?pi=18
http://countryporchhomedecor.com/donna-sharp/lodge-quilts/king%2C-woodland
http://www.countryporchhomedecor.com/park-designs/accessories/home-place-tumbler
http://countryporchhomedecor.com/park-designs/rag-rugs/redmon-rag-rug-36x6Any ideas?
Also, it seems like it's not honoring the rel-canonical tag. It keeps saying that pages with a rel canonical tag are duplicates when some of the urls that it's flagging shouldn't even be indexed because of the canonical tag. The "pi" in the query string should not be indexed!
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=3
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=6
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=7
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=6
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=10
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=8
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=8
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=7
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=7
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=1
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=8
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=5
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=10
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=3
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=5
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=4
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=9
http://www.countryporchhomedecor.com/bedding-%26-quilts/shams/standard-shams
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=1
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=6
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=1
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=5
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=2
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=9
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=4
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=3
http://countryporchhomedecor.com/bedding-%26-quilts/shams/standard-shams
http://www.countryporchhomedecor.com/bedding-%26-quilts/shams/standard-shams?pi=18
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=9
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=10
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=2
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=2
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=4 -
No problem! Yes, same is true for HTTP and HTTPS
-
Is is the same way with https and non https pages? Should only one of those be accessible per page?
-
Ok thank you!
-
That is correct, you should be using rel=next/prev for markup on paginated sections. But after noticing the www and non-www issue, I don't think your problem is related to canonicals or prev/next.
Regardless of what you're doing with canonical tags or prev/next, you pages should never be accessible at both www and non-www versions. You're going to be at a duplicate content risk as long as both versions exist.
-
Thank you very much for your response! On the paginated pages I don't think you're supposed to use the canonical tag. Instead you're supposed to use the next/prev tag which is what I did. the next/prev tag points only to pages without query string values and those are the pages that are supposed to be indexed. So there shouldn't be individual pages that are separated by query string values right? They should all use non query string value pages.
Even though I do have both www and non www pages accessible, on all of the pages, I am either using the canonical tag or the next/prev tag on paginated pages. Shouldn't that tell search engines which to index??
-
Hi,
Regarding the first set of URLs: I took a look at a handful of those URLs, and it's entirely possible that you're getting duplicate notices on those. Rogerbot flags any 2 pages as duplicates if the source code of those URLs matches at 90% or more. So it's not identical, but not different enough that search engines can discern. Most of the products you've listed there have no content, or a very small amount, meaning that when you consider the rest of the code involved with that page, it mostly matches the homepage.
Regarding the second set: I ran those URLs through Screaming Frog and don't see any canonical tags. Keep in mind, just because URLs aren't indexed in search engines, doesn't mean Rogerbot doesn't have access to them.
*Update - on further digging, I think I found the source of all of your duplicate issues. Both www and non-www versions of your URLs are accessible. One of them should redirect to the other, doesn't matter which, but both should not render.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate URLs
A campaign that I ran said that my client's site had some 47,000+ duplicate pages and titles. I was wondering how I can possibly set that many 301 redirects, but a Moz help engineer said it has a lot to do with session IDs. See this set of duplicate URLs: http://www.lumberliquidators.com/ll/c/engineered-hardwood-flooring (clearly the main URL for the page)
Moz Pro | | AlanJacob
http://www.lumberliquidators.com/ll/c/engineered-hardwood-flooring?PIPELINE_SESSION_ID=0ac00a2e0ad53eb90cb0b0304d178fc1
http://www.lumberliquidators.com/ll/c/engineered-hardwood-flooring?PIPELINE_SESSION_ID=0ac3039d0ad4af2720b3ccd2238547ab
http://www.lumberliquidators.com/ll/c/engineered-hardwood-flooring?PIPELINE_SESSION_ID=0ac071ed0ad4af292684b0746931158f To a crawler, that looks like 4 different pages, when it's clear that they're actually all different URLs for the same page. I was wondering if some of you, maybe with experience in site architecture, would have insight into how to address this issue? Thanks Alan0 -
Competitor Ranks Highly Despite LOTS of Duplication
One of my Moz campaigns tracks my main competitor. I can't get over the fact that this competitor continues to rank well despite having thousands of duplication issues while I currently have less than 10. Our websites are similar both structurally and product-wise. The two images attached include OSE comparisons for the keyword "kraft envelopes". My competitor ranks #2 for the term, while I rank #9. I don't get why! And again, looking from a top-level perspective, MOZ shows that my competitor's website is full of duplication issues and other problems while my site is healthy as a horse. All responses are greatly appreciated. QmSlfET.jpg 9CDny60.jpg
Moz Pro | | jampaper0 -
Issue getting total links, page & domain authority
Hi guys, I am trying to get total links, page & domain authority using the API. I am requesting the following columns: Cols=6871947673632768328204816384343597383681653687091214 { "fjid": 207343179, "ued": 43324279, "pib": 255645, "ptrr": 0.0056131743357352125, "fmrp": 8.246626591590841, "unid": 954915, "fjf": 4003651, "fjr": 0.00040067116628622016, "ftrp": 8.308303969566644, "ftrr": 0.0012189619975325583, "fejp": 9.265328830369816, "pnid": 45883246, "fjd": 2480265, "ujfq": 1277385, "pjip": 1230240, "fjp": 9.586342983782004, "fuid": 294877628, "uu": "www.google.com/", "pejr": 0.0004768398971363439, "ufq": "www.google.com/", "pejp": 9.647424778525615, "ujp": 300689, "utrp": 7.916901429865898, "ptrp": 9.487254666203722, "utrr": 0.001639219985667878, "fmrr": 0.000731592927123369, "pda": 100, "pjd": 5600882, "ulc": 1342758719, "fnid": 12165784, "fejr": 0.00016052965883996156, "ujb": 107264 } I cannot see the UPA column returned in the JSON object. Im using 34359738368 for the UPA column. I need to retrieve the three fields (page authority, domain authority and total links) in the same query. Is it possible?
Moz Pro | | Srvwiz0 -
Port 80 and Duplicate Content
The SEOmoz Web App is showing me that every single URL on one of my clients' domains has a duplicate in the form of the URL + :80. For instance, the app is showing me that www.example.com/default.aspx is duplicated in the form of www.example.com:80/default.aspx Any idea if this is an actual problem or just some kind of reporting error? Any help would be appreciated.
Moz Pro | | AnthonyMangia0 -
Duplicate Content
I have tried searching for an exact example of the issues I am seeing, but didn't come up with anything. I decided to post my own question so I can get a direct answer on what I am experiencing. I recently took over a website and its' existing SEO practices with it. Upon placing the site on SEOmoz, I received many (LOTS) of duplicate content warnings. Pretty much, this is how the website is setup: domain.com/keyword-is-here/ but it is also coming up as domain.com/keyword-is-here/index.htm - Should I setup a redirect so domain.com/keyword-is-here/index.htm points to domain.com/keyword-is-here.htm or should I just leave it alone since it's pointing to the same exact? Any information on this questions is greatly appreciated in advance.
Moz Pro | | EQ-Richie0 -
Some questions on Canonical tag AND 301 redirect
Hi everyone, I'm new here - always loved SEOMoz and glad to be part of the Pro community now. I have 2 questions regarding the Canonical URL tag. Some background info: We used to run an OsCommerce store, and recently migrated to Magento. In doing so, we right away created 301 redirects of the old category pages (OsCommerce) to the new category pages (Magento) via the Magento admin. Example: www.example.com/old-widget-category.html
Moz Pro | | yacpro13
301 redicrected to
www.example.com/new-widget-category.html In Magento admin, we have enabled the Canonical tag for all product and category pages. Here's how Magento sets up the Canonical tag: The URL of interest which we want to rank is:
www.example.com/new-widget-category.html However Magento sets up the canonical tag on this page to point to:
www.example.com/old-widget-category.html When using the SEOMoz On Page Report Card, it pick this up as an error because the Canonical tag is pointing to a different URL. However, if we dig a little deeper, we see that the URL being pointed to
www.example.com/old-widget-category.html
has a 301 redirect to
www.example.com/new-widget-category.html
which is the URL we wan to rank. So because we set up a 301 redirect of the old-page to the new-page, on the new-page the canonical tag points to the old-page. Question 1)
What are you opinions on this? Do you think this method of setting up the Canonical tag is acceptable? Second question... We use pagination for category pages, so if we have 50 products in one category, we would have 5 pages of 10 products. The URL's would be: www.example.com/new-widget-category.html (which is the SAME as ?p=1)
www.example.com/new-widget-category.html?p=1
www.example.com/new-widget-category.html?p=2
www.example.com/new-widget-category.html?p=3
www.example.com/new-widget-category.html?p=4
www.example.com/new-widget-category.html?p=5 Now ALL the URLs above have the canonical tag set as:
<link rel="canonical" href="http://www.example.com/new-widget-category" /> However, the content of each page (page 1, 2, 3, 4, 5) is different because different products are displayed. So far most what I read regarding the Canonical tag is that it is used for pages that have the same content but different URLs. I would hope that Google would combine the content of all 5 pages and view the result as a single URL www.example.com/new-widget-category Question 2) Is using the canonical tag appropriate in the case described above? Thanks !0 -
Getting PA & DA off of a list of links
I have a list of links that I want to get PA and DA for each individual link, can this be done in some way other than one at a time? I've heard this can be done with excel and using api but I don't know the specifics of this.. Help would be appreciated
Moz Pro | | Fergclaw2