Duplicate URL errors when URL's are unique
-
Hi All,
I'm running through MOZ analytics site crawl report and it is showing numerous duplicate URL errors, but the URLs appear to be unique. I see that the majority of the URL's are the same, but shouldn't the different brands make them unique to one another?
http://www.sierratradingpost.com/clearance~1/clothing~d~5/tech-couture~b~33328/
http://www.sierratradingpost.com/clearance~1/clothing~d~5/zobha~b~3072/
Any ideas as to why these would be shown as duplicate URL errors?
-
There is long article on the dev blog how they determine whether pages are duplicates - check https://moz.com/devblog/near-duplicate-detection/ - it's quite technical stuff - but this is the part which might interest you:
"This leads to one of the questions we get asked a lot: Why do I see duplicate content warnings in the context of Custom Crawl for pages that I see as different. Ultimately, it’s always because of the same reason: because no dechroming is done, there is a small amount of unique content relative to the total content. One of the places where this crops up a lot is web stores, where there’s a large amount of chrome layout, but only a short product description associated with it."
Dechroming : removing things like navigation, footer, ..etc from the page (exact def. to be found in the article)
If you compare both pages - apart from the image & product title there isn't too much difference between them so the crawler sees only a very small % of content which is different and marks them as duplicates.
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Avoid landing page redirects C (75) SERVER HIGH What's this mean?
HI we have a wordpress website with a chain of redirected urls? one is with www. the other without? How do I found out where this is? or what could be causing it? Any help much appreciated 🙂
On-Page Optimization | | KellyDSD861 -
Internal 404 Error
Hi sorry for the newbie question, I have a few 404 pages on my moz crawl report. so for example this one : http://www.dwliverpoolphotography.co.uk/blog/www.coraclecomm.wordpress.com. How can I find the page that is linking to it so I can fix the link or delete it? Best wishes. David.
On-Page Optimization | | WallerD0 -
Duplicate Page content | What to do?
Hello Guys, I have some duplicate pages detected by MOZ. Most of the URL´s are from a registracion process for users, so the URL´s are all like this: www.exemple.com/user/login?destination=node/125%23comment-form What should I do? Add this to robot txt? If so how? Whats the command to add in Google Webmaster? Thanks in advance! Pedro Pereira
On-Page Optimization | | Kalitenko20140 -
Tag-URLs in Magento
Hello, I have got a problem concerning Tag-URLs in Magento (the URLs mentioned are just fictitious 😞 At the moment, they look something like this: (1) http://store.com/tag/product/list/tagId/1/ ... so these URLs are not search engine friendly at all. Using a Magento extension you could transform them in speaking URLs: (2) http://store.com/tag/digital-cameras What would you do if you sold, say, digital cameras and your online shop ranked high for the keyword "digital camera" with URL No. 1 (not search engine friendly). Would you transform (1) in (2) and 301 all non speaking URLs? But would you keep the high ranking for "digital camera" when 301 to URL No. (2). But, what I'm most concerned of is : There is actually a landing page (category page) for the keyword "digital camera" : http://store.com/digital-cameras. Shouldn't the last URL rank high for "digital camera"? (instead of the tag URLs). But given the situation above, does it make sense now to 301 the tag URL to the category page? I would perhaps lose my good ranking, wouldn't I? Thanks a lot for your help! Martin
On-Page Optimization | | SmartyMarty810 -
Wordpress pages URL's redirection.
I was checking W3C Markup Validation and in report it was shown that that pages (not post or any other URL's just PAGES) at investmentcontrarians.com are 301 redirected. e.g. original URL "http://www.investmentcontrarians.com/debt-crisis" which is redirected to "http://www.investmentcontrarians.com/debt-crisis/" I know that its not that serious issue, but still want to know why only pages are being redirected and how can we avoid it.
On-Page Optimization | | NumeroUnoWebSolutions0 -
Question about URLs
Hello! I have a client that wants to upload an URL like this: www.example.com/keyword/page-name.html The main problem is that www.example.com/keyword/ doesn't exist and gives a 404 error so I'd prefer not doing that...... What do you think about this? And if the client wants to go ahead, is there any solution? A 301 to the final page would help? Thank you in advance!
On-Page Optimization | | Juandbbam0 -
Would it be bad to change the canonical URL to the most recent page that has duplicate content, or should we just 301 redirect to the new page?
Is it bad to change the canonical URL in the tag, meaning does it lose it's stats? If we add a new page that may have duplicate content, but we want that page to be indexed over the older pages, should we just change the canonical page or redirect from the original canonical page? Thanks so much! -Amy
On-Page Optimization | | MeghanPrudencio0 -
Duplicate content Issue
I'm getting a report of duplicate title and content on: http://www.website.com/ http://www.website.com/index.php Of course, they're the same pages but does this need to be corrected somehow. Thanks!
On-Page Optimization | | dbaxa-2613380