Duplicate URL errors when URL's are unique
-
Hi All,
I'm running through MOZ analytics site crawl report and it is showing numerous duplicate URL errors, but the URLs appear to be unique. I see that the majority of the URL's are the same, but shouldn't the different brands make them unique to one another?
http://www.sierratradingpost.com/clearance~1/clothing~d~5/tech-couture~b~33328/
http://www.sierratradingpost.com/clearance~1/clothing~d~5/zobha~b~3072/
Any ideas as to why these would be shown as duplicate URL errors?
-
There is long article on the dev blog how they determine whether pages are duplicates - check https://moz.com/devblog/near-duplicate-detection/ - it's quite technical stuff - but this is the part which might interest you:
"This leads to one of the questions we get asked a lot: Why do I see duplicate content warnings in the context of Custom Crawl for pages that I see as different. Ultimately, it’s always because of the same reason: because no dechroming is done, there is a small amount of unique content relative to the total content. One of the places where this crops up a lot is web stores, where there’s a large amount of chrome layout, but only a short product description associated with it."
Dechroming : removing things like navigation, footer, ..etc from the page (exact def. to be found in the article)
If you compare both pages - apart from the image & product title there isn't too much difference between them so the crawler sees only a very small % of content which is different and marks them as duplicates.
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Making Shopify URL's Simpler - Losing the words 'collection', 'product' and 'page' in a Shopify store URL. Any advice?
Hi Mozers! I have a Shopify store (of which there are many advantages) however one big SEO disadvantage, is that my URL structures contravene all Moz advice on dynamic URL structure and whats more I am reminded about this every week when I have a Moz site crawl and I have a batch of URL's that are longe than the 75 characters. A Shopify URL will run www.domain name.com/collections/collection-name/product/product-name. According to advice a it should be www.domain name.com/collection-name/product-name - Don't even get started on sub-collections! I sell portfolio books, album etc and keepsake memory boxes (so long keywords) AND, I have a long(ish) business name. So, For user experience and keyword length, do I just ignore trying to achieve a dynamic URL under 75 characters? When I have asked Shopify, the say their URL's are an integral part of the "Ruby on Rails" system, so nothing can be done Or can it ??? I can't be the only Moz member with this issue can I ??
On-Page Optimization | | nick_HandCo0 -
Single Page on my client's website is not crawling and indexing new changes. What could be the possible reason?
I made several changes on client's website on different pages, changed titles, add content on few pages, moved blog from subdomain to sub directory. Everything is crawled but there is one page on the website (not part of the blog) that isn't getting crawled in Google and picking up changes. The last crawl of the website is 2 days back whereas that page was last crawled on 30th sep. I just wanted to know the possible reasons and has anyone encountered this before?
On-Page Optimization | | MoosaHemani0 -
How do you handle URLs with slashes?
I asked this question before, but with a different scenario. I upgraded my plan to a more advanced cart and all of my URLs changed about 1.5 years ago. I knew nothing about redirects and such, so none of that was done. Basically, let's say my site was: http://www.abc.com, but when people actually visit my site, they are directed to https://www.abc.com/. I have asked my host about redirecting and she that it is not possible. In the past, the link shared has been just www.abc.com . Will this hurt my ranking? My second question is ...let's say I have a link http://www.abc.com/blog , but now, the link is http://www.abc.com/blog/ . Will I be affected, since all my old links omit the slash?
On-Page Optimization | | tiffany11030 -
Competitor's 'hidden' links harming my site?
Hi everyone, I'm new to both Moz & seo, and am attempting to tackle our site's issues after being hit by panda / penguin, so would be grateful for any advice offered. I bought a website 3 years ago after the previous company that ran it went into administration. Having bought the website, it became apparent that the employees of the previous company had copied the entire site content, and relaunched it with a new look / brand. Over the last 3 years they've rewritten much of the content, but there remains a lot of links from their site back to ours which have had the anchor text stripped out, and point to images on our site which have since been removed, example below... <a href="http://www.MyCompany.com/catalog/images/filename.pdf" target="<a class="attribute-value">_blank</a>"><strong>strong>a> What I'm trying to understand is whether the 404 errors being returned by the broken links, and the presence of 'hidden' links on their site, is likely to reflect badly on our site or theirs? I'm not interested in outing anyone here, and I realise the standard recommendation for these kinds of situations is to write to the company telling them to remove the offending content, but if at all possible I'd prefer to fix our site by improving content & links etc, rather than 'force' them to take action and inadvertently improve their own site's content / rankings. As I say, all advice gratefully received 🙂
On-Page Optimization | | Sandy_M0 -
Blog URL
I know that this question has been asked in the past, and that website.com/blog is better for seo purposes than blog.website.com. We want to setup a custom blog on our site, using Wordpress. Our designers/host are telling us that buy using website.com/blog can causes issues b/c Wordpress is open source, and our site could be hacked? Is there anything we should do about this? Any suggestions? Any Advice appreciated!!! Thanks!
On-Page Optimization | | TP_Marketing0 -
Duplicate content on homepage?
Hi I have just created a new campaign and it states that I have duplicate page content which would affect search rankings. Basically it is counting my site www.mydomain.com and www.mydomain.com/index.php as two seperate pages. How can I make it so that only www.mydomain.com is visible reducing the duplicate content issue? Many Thanks
On-Page Optimization | | idv0 -
Seeking URL Advice
Hey Moz Community, I'm looking for some URL structure advice for a new directory of a website. We're trying to rank for the term 'internships abroad in <country>'</country> We have roughly 100 pages targeting specific countries. Right now the URL structure is www.gooverseas.com/internships-abroad/china, but some of my colleagues believe this structure would be better: www.gooverseas.com/internships-abroad/intern-in-china. I personally prefer the shorter structure, but we couldn't come to any agreement so we thought we'd pose the question to the community. Any thoughts? Thanks!
On-Page Optimization | | dunklea0 -
Are duplicate titles an issue for pages I don't need ranking for?
A client has a load of duplicate page titles on their site. However, to cut a long story short, most of these pages are pointless and therefore we don't need ranking for them. As such, I'm not concerned whether any of the pages with duplicate content on them are ranked or not..... unless having duplicate page titles / content on these pages could mean that other pages on the site, like the homepage, don't rank as high because of this. Do I need to worry about duplicate titles on these pages, or can I ignore duplicate content on pages that I don't want to be ranked? Hope that makes sense!
On-Page Optimization | | RiceMedia0