Duplicate URL errors when URL's are unique
-
Hi All,
I'm running through MOZ analytics site crawl report and it is showing numerous duplicate URL errors, but the URLs appear to be unique. I see that the majority of the URL's are the same, but shouldn't the different brands make them unique to one another?
http://www.sierratradingpost.com/clearance~1/clothing~d~5/tech-couture~b~33328/
http://www.sierratradingpost.com/clearance~1/clothing~d~5/zobha~b~3072/
Any ideas as to why these would be shown as duplicate URL errors?
-
There is long article on the dev blog how they determine whether pages are duplicates - check https://moz.com/devblog/near-duplicate-detection/ - it's quite technical stuff - but this is the part which might interest you:
"This leads to one of the questions we get asked a lot: Why do I see duplicate content warnings in the context of Custom Crawl for pages that I see as different. Ultimately, it’s always because of the same reason: because no dechroming is done, there is a small amount of unique content relative to the total content. One of the places where this crops up a lot is web stores, where there’s a large amount of chrome layout, but only a short product description associated with it."
Dechroming : removing things like navigation, footer, ..etc from the page (exact def. to be found in the article)
If you compare both pages - apart from the image & product title there isn't too much difference between them so the crawler sees only a very small % of content which is different and marks them as duplicates.
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Making Shopify URL's Simpler - Losing the words 'collection', 'product' and 'page' in a Shopify store URL. Any advice?
Hi Mozers! I have a Shopify store (of which there are many advantages) however one big SEO disadvantage, is that my URL structures contravene all Moz advice on dynamic URL structure and whats more I am reminded about this every week when I have a Moz site crawl and I have a batch of URL's that are longe than the 75 characters. A Shopify URL will run www.domain name.com/collections/collection-name/product/product-name. According to advice a it should be www.domain name.com/collection-name/product-name - Don't even get started on sub-collections! I sell portfolio books, album etc and keepsake memory boxes (so long keywords) AND, I have a long(ish) business name. So, For user experience and keyword length, do I just ignore trying to achieve a dynamic URL under 75 characters? When I have asked Shopify, the say their URL's are an integral part of the "Ruby on Rails" system, so nothing can be done Or can it ??? I can't be the only Moz member with this issue can I ??
On-Page Optimization | | nick_HandCo0 -
I have an eCommerce Site with in some cases, 100s of versions of the same product. How do I avoid "duplicate content" without writing literally 100s of unique product descriptions for the exact same product?
For instance, one item where the only difference is the Sports Team Logo is different, etc... or It comes in a variety of color Variants. I'm using Shopify.
On-Page Optimization | | pstone291 -
Strange error on my website
Hi Wondering if anyone could help out here please. I tried to crawl my website using screaming frog and got a 500 error warning (odd considering it worked yesterday). I tried google speed test and got the following message - PageSpeed Insights received a 500 response from the server. To analyze pages behind firewalls or that require authentication. However, it also crawled the site and gave me a score of 70/100. It showed the site thumbnail, as normal. This was updated this morning and it showed the latest one so must have crawled. I can access the site, as can my 'people', the site is getting hits from real users in Google Analytics so must be working. Am very puzzled how the site can be working and not be working at the same time! If possible i would rather not share the url. The site is a test of concept and is very messy looking. it is more a test to see what happens if i do x... Many thanks Carl
On-Page Optimization | | daedriccarl0 -
Canonical URL Tag
Hi, I have two pages that are identical on my site: http://www.absolutepower.nl/creatine-monohydraat and http://www.absolutepower.nl/CREATINE/creatine-monohydraat Should I use the canonical URL tag in this case? Thanks, Jasper
On-Page Optimization | | Japking0 -
I have one page on my site... but still get duplicate name and content errors.
i have only the index.html page. my domain has a permanent 301 to the root. why am i getting duplicate problems? i only have one page the index .html???
On-Page Optimization | | one4u2see0 -
Does a page's url have any weight in Google rankings?
I'm sure this question must have been asked before but I can't find it. I'm assuming that the title tag is far more important than the page's url. Is that correct? Does the url have any relevance to Google?
On-Page Optimization | | rdreich490 -
Trailing slash on URLs
Hi everyone My question is regarding trailing URLs in Wordpress A designer setup a site and made the URL structure something like: www.website.com/description/ but I want to change it to www.website.com/description as this is more SEF A trailing / was added to the permalink structure in Wordpress, if I change this will google see these as new URLs and will all the current URLs become 404? Or should I just leave it? Thanks in advance
On-Page Optimization | | webseoservices0 -
Does google treat all urls equal?
Sorry for the lame title, i couldn't think of a better one. I want to know if google treats this: http://www.domain.com/products/some-product-name the same as it would treat: http://www.domain.com/?products=some-product-name if not, could you tell me the differences?
On-Page Optimization | | adriandg0