Duplicate Content Issue in Magento
-
Hi I need help in resolving the duplicate content issue on my magento site I got a product
My main product url is https://www.oakfurnitureking.co.uk/shop-by-product/boston-solid-oak-4-drawer-chest and it got variation of url see below that are causing duplicate content issue , I have inserted the canonical tag on the below url and my main url is https://www.oakfurnitureking.co.uk/shop-by-product/boston-solid-oak-4-drawer-chest but still moz is showing it as duplicate content.
Help Please
<colgroup><col width="1003"></colgroup>
| https://www.oakfurnitureking.co.uk/product/oak-bedroom-furniture/boston-solid-oak-4-drawer-chest |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/6/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/17/ |
| https://www.oakfurnitureking.co.uk/shop-by-range/boston/boston-solid-oak-4-drawer-chest |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/42/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/63/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/67/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/46/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/79/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/88/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/75/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/90/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/92/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/33/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/27/ |
| https://www.oakfurnitureking.co.uk/shop-by-range/boston-solid-oak-4-drawer-chest |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/50/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/22/ |
| https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/74/ | -
Roger is plain smart. Not wise.
-
Hi Adnan,
remember that the rel="canonical" is not a directive, hence it is not saying "Bots don't index this URL, index this other one", but "Could you bots don't index this URL, and instead index this other one?".
The canonical tag is a suggestion, that usually Google, Bing and Yahoo! respect.
But the canonicalized URLs still are crawled... and for that reason Roger Mozbot alerts you about the duplicate content issue (as well as it notices you that they have the canonical tag set up.
If you know that they are ok, simply don't care of them.
But knowing how Mozbot has a limited budget crawl, if you really want the best from its crawling service, disallow just for it those kinds of duplicates via robots text.
-
If you check your Robots.txt https://www.oakfurnitureking.co.uk/robots.txt You will see the fallow lines:
## Do not crawl seach pages and not-SEO optimized catalog links Disallow: /catalogsearch/ Disallow: /catalog/product_compare/ Disallow: /catalog/category/view/ Disallow: /catalog/product/view/ Also if you Google site:https://www.oakfurnitureking.co.uk/catalog/product/view/id/45/s/boston-solid-oak-4-drawer-chest/category/6/ Your site do not show up... So you are GOOD.
-
You need to stop Moz's crawler from crawling these potential duplicate pages. You need not worry as Google will not index them. What Moz tells you is that if you do not wish to index these pages, block it from crawlers. No point in the crawlers crawling these pages anyway.
If you email me direct, I can send you a standard robots.txt that we use to block out the crawlers from most potential duplicate and unnecessary pages.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Page Content
Hey Moz Community, Newbie here. On my second week of Moz and I love it but have a couple questions regarding crawl errors. I have two questions: 1. I have a few pages with duplicate content but it say 0 duplicate URL's. How do I know what is duplicated in this instance? 2. I'm not sure if anyone here is familiar with an IDX for a real estate website. But I have this setup on my site and it seems as though all the links it generates for different homes for sale show up as duplicate pages. For instance, http://www.handyrealtysa.com/idx/mls...tonio_tx_78258 is listed as having duplicate page content compared with 7 duplicate URLS: http://www.handyrealtysa.com/idx/mls...tonio_tx_78247
On-Page Optimization | | HandyRealtySA
http://www.handyrealtysa.com/idx/mls...tonio_tx_78253
http://www.handyrealtysa.com/idx/mls...tonio_tx_78245
http://www.handyrealtysa.com/idx/mls...tonio_tx_78261
http://www.handyrealtysa.com/idx/mls...tonio_tx_78258
http://www.handyrealtysa.com/idx/mls...tonio_tx_78260
http://www.handyrealtysa.com/idx/mls...tonio_tx_78260 I've attached a screenshot that shows 2 of the pages that state duplicate page content but have 0 duplicate URLs. Also you can see somewhat about the idx duplicate pages. rel="canonical" is functioning on these pages, or so it seems when I view the source code from the page. Any help is greatly appreciated. skitch.png0 -
Duplicate content on partner site
I have a trade partner who will be using some of our content on their site. What's the best way to prevent any duplicate content issues? Their plan is to attribute the content to us using rel=author tagging. Would this be sufficient or should I request that they do something else too? Thanks
On-Page Optimization | | ShearingsGroup0 -
Site Duplicated despte redirect
Buon pormeriggio from I can smell Whaler Chips Through the window Wetherby,
On-Page Optimization | | Nightwing
When you Google Thakray Medical Museum 2 sites appear in the SERPS, yikes! Now the .org site is no longer hosted & point to the .co.uk site when clicked on but in a nutshell I wantto get rid of the .org site
as illustrated here: http://s216.photobucket.com/user/zymurgy_bucket/media/two-versions-same-website-yikes_zps182e6e12.jpg.html Actions taken so far:
1: Wembaster tools re index request for the .co.uk site
2: Redirect configured to point .org site to the .co.uk What else is left apart from updating the xml site but ultimating i do not want to see the the .org site but it doesnt exist (well id did a few month back but is no longer hosted so i am told) Any insights welcome,
GRazie tanto,
David0 -
Duplicate content
crawler shows following links as duplicate http://www.mysite.com http://mysite.com http://www.mysite.com/ http://mysite.com. http://mysite.com/index.html How can i solve this issue?
On-Page Optimization | | bhanu22170 -
Duplication About PDF Files on Website
Hello, My site's URL (web address) is: http://www.vostastores.com/ Above is the Our Website URL. We are in the process of Upgrading Our Website and for that we are adding all Details of each and every products. One of the thing that we are planning to do is to get Manufacturer's product PDF files on our Website which the manufacturer already have on their website. So our Question is that Since the manufacturer has the file on their website and we want to add the same on our website, Will be there any Duplication issue? If yes, then please provide us with a Solution by which we can add the same on our website. Thanks & Regards.
On-Page Optimization | | CommercePundit0 -
Numbers above actual site content
Most pages on my website contain many numbers above the actual text on the page. This is useful for users and looks good on an actual view of the page. However, when a bot reads the page it appears as rows of numbers with a few sentences at the bottom of the page. Does having these number have a negative SEO effect? If so, should I change them to something such as an image so they aren't readable by search engines?
On-Page Optimization | | theLotter0 -
Duplicate content on homepage?
Hi I have just created a new campaign and it states that I have duplicate page content which would affect search rankings. Basically it is counting my site www.mydomain.com and www.mydomain.com/index.php as two seperate pages. How can I make it so that only www.mydomain.com is visible reducing the duplicate content issue? Many Thanks
On-Page Optimization | | idv0 -
Duplicate Page Titles?
I'm running a campaign report within SEOmoz & am getting 9 pages that appear on this report. They all happen to be our author pages www.example.com//author/admin We have multiple authors. Is there a proper way that I should take care of this? Also as a side note, I'm using Yoast Wordpress SEO plugin, is there a setting on their I should change that will fix this issue? Or is it an issue at all? Thanks, BJ
On-Page Optimization | | seointern0