Duplicate URL errors when URL's are unique
-
Hi All,
I'm running through MOZ analytics site crawl report and it is showing numerous duplicate URL errors, but the URLs appear to be unique. I see that the majority of the URL's are the same, but shouldn't the different brands make them unique to one another?
http://www.sierratradingpost.com/clearance~1/clothing~d~5/tech-couture~b~33328/
http://www.sierratradingpost.com/clearance~1/clothing~d~5/zobha~b~3072/
Any ideas as to why these would be shown as duplicate URL errors?
-
There is long article on the dev blog how they determine whether pages are duplicates - check https://moz.com/devblog/near-duplicate-detection/ - it's quite technical stuff - but this is the part which might interest you:
"This leads to one of the questions we get asked a lot: Why do I see duplicate content warnings in the context of Custom Crawl for pages that I see as different. Ultimately, it’s always because of the same reason: because no dechroming is done, there is a small amount of unique content relative to the total content. One of the places where this crops up a lot is web stores, where there’s a large amount of chrome layout, but only a short product description associated with it."
Dechroming : removing things like navigation, footer, ..etc from the page (exact def. to be found in the article)
If you compare both pages - apart from the image & product title there isn't too much difference between them so the crawler sees only a very small % of content which is different and marks them as duplicates.
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should I change my website urls?
We're translating our website in a few languages (FR / DE / JP) using subdirectories. So our website will have the following urls www.brand.com/en
On-Page Optimization | | dcalexandra
www.brand.com/fr
www.brand.com/de
www.brand.com/jp I would like to change the url structure of a few pages from www.brand.com/section/feature-name to www.brand.com/feature-name Is it a good idea to do this now since we're adding the subfolders and these are anyway new urls in google's eyes?0 -
Hello, I've heard that the outbound links I provide in my content should have a high degree of relevancy to the topic I'm writing about or they aren't really worth including. Is this true?
Hello, I've heard that relevancy of the content between the source page and the target page of outbound links in my content matters greatly. The outbound links I provide in my content should have a high degree of relevancy to the topic I'm writing about, or they aren't really worth including. Example: Don't just link to the homepage of an organization mentioned in the article, link to a page on their site that is related to the topic you are writing about. Is this true? Would including less relevant links negatively impact SEO in any way?
On-Page Optimization | | DJBKBU0 -
Crawl errors
Hi I have the following errors on my site and was wondering would it help improve my ranking to fix : Missing Meta Description Tag 137Duplicate Page Title 17Title Element is Too Long 6Temporary Redirect 3
On-Page Optimization | | WallerD0 -
Duplicate Page Content
Hi there, We keep getting duplicate page content issues. However, its not actually the same page.
On-Page Optimization | | HamiltonIsland
E.G - There might be 5 pages in say a Media Release section of the website. And each URL says page 1, 2 etc etc. However, its still coming up as duplicate. How can this be fixed so Moz knows its actually different content?0 -
Locating Duplicate Pages
Hi, Our website consists of approximately 15,000 pages however according to our Google Webmaster Tools account Google has around 26,000 pages for us in their index. I have run through half a dozen sitemap generators and they all only discover the 15,000 pages that we know about. I have also thoroughly gone through the site to attempt to find any sections where we might be inadvertently generating duplicate pages without success. It has been over six months since we did any structural changes (at which point we did 301's to the new locations) and so I'd like to think that the majority of these old pages have been removed from the Google Index. Additionally, the number of pages in the index doesn't appear to be going down by any discernable factor week on week. I'm certain it's nothing to worry about however for my own peace of mind I'd like to just confirm that the additional 11,000 pages are just old results that will eventually disappear from the index and that we're not generating any duplicate content. Unfortunately there doesn't appear to be a way to download a list of the 26,000 pages that Google has indexed so that I can compare it against our sitemap. Obviously I know about site:domain.com however this only returned the first 1,000 results which all checkout fine. I was wondering if anybody knew of any methods or tools that we could use to attempt to identify these 11,000 extra pages in the Google index so we can confirm that they're just old pages which haven’t fallen out of the index yet and that they’re not going to be causing us a problem? Thanks guys!
On-Page Optimization | | ChrisHolgate0 -
Keyword in url, which way better?
Hello, is there a difference between urls for targeting keyword "brazil tourist visa" fastbrazilvisas.com/tourist or fastbrazilvisas.com/brazil-tourist-visa ? ran the report In-Page Optimization it tells "no keyword usage in url". is there an idea behind that? thanks
On-Page Optimization | | Kotkov0 -
The URL Inside
Howdy SEO'ers, I have a quick question for the SEO gurus out there. When constructing "better" search friendly URLs would one of these be better than the other? Example 1: http://Domain.com/Category/Sub-Category/Product-name Example 2: http://Category.Domain.com/Sub-Category/Product-name In this example the category could be phones and the sub-category brands of phones. Is either one of these URLs "better" than the other in terms of ranking? Thanks! I'll hang-up and listen to your answer. 🙂 Jonathan
On-Page Optimization | | creativedepartment0 -
Not making a change of the 100's in crawl Diagnostic
Based on the PRO crawl Diagnostics – if we don’t make a change on 1 page, does that just affect the SEO on that one page, or does it affect the SEO on all pages of the site? E.g. If we get a “Too many on page links” for a certain page that we don’t really want to rank for – does not fixing that particlaur page affect the site as a whole? Hope I explained this ok..
On-Page Optimization | | inhouseninja0