Duplicate URL errors when URL's are unique
-
Hi All,
I'm running through MOZ analytics site crawl report and it is showing numerous duplicate URL errors, but the URLs appear to be unique. I see that the majority of the URL's are the same, but shouldn't the different brands make them unique to one another?
http://www.sierratradingpost.com/clearance~1/clothing~d~5/tech-couture~b~33328/
http://www.sierratradingpost.com/clearance~1/clothing~d~5/zobha~b~3072/
Any ideas as to why these would be shown as duplicate URL errors?
-
There is long article on the dev blog how they determine whether pages are duplicates - check https://moz.com/devblog/near-duplicate-detection/ - it's quite technical stuff - but this is the part which might interest you:
"This leads to one of the questions we get asked a lot: Why do I see duplicate content warnings in the context of Custom Crawl for pages that I see as different. Ultimately, it’s always because of the same reason: because no dechroming is done, there is a small amount of unique content relative to the total content. One of the places where this crops up a lot is web stores, where there’s a large amount of chrome layout, but only a short product description associated with it."
Dechroming : removing things like navigation, footer, ..etc from the page (exact def. to be found in the article)
If you compare both pages - apart from the image & product title there isn't too much difference between them so the crawler sees only a very small % of content which is different and marks them as duplicates.
Dirk
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does Google's algo look at all traffic mediums with regs to onpage metrics or only organic traffic metrics?
Hi folks, This is something I've pondered for a while. I've ask a couple of Googlers but no reponse yet and I don't I'll get one! In your opinion, do you think Google looks at on page metrics like bounce rate for example from all traffic mediums (organic, paid, email, social referral etc etc) or they only look at on page metrics from organic traffic? I'm not talking about direct correlations from other mediums. I'm only talking about when a user lands on a website, do the actions they take matter with regards to Google's search algo no matter of the referring medium, or do Google only look at onpage metrics on visits which came to the site via organic search as a medium. Option 1 As a very simplified example: Google gives extra weight in the SERPs to website A which has an average bounce rate of 30% from all mediums compared to website B which has a bounce rate of 50% from all mediums. Option 2 Google gives extra weight in the SERPs to website A which has an average bounce rate of 30% from organic traffic only compared to website B which has a bounce rate of 50% from organic traffic only. I'm not sure if anyone outside Google has the answer/proof of this but was keen to get other people's thoughts. If you think the also uses one or the other, can you give an insights/proof of one or the other? For me it would make sense for them only to use onpage metrics from sessions which came from organic seach traffic, but who knows! Merci buckets, Gill.
On-Page Optimization | | Cannetastic0 -
Better to hyphenate URL or no?
Sea Glass Jewelry or Sea-Glass-Jewelry My domain name does not have my keyword in it, so I have been using the category as a means to get the keyword in the URL. My site would say www.abcdefghijk.com/sea-glass-jewelry/sterling-starfish-necklace When I run the review, it tells me that I have too many parameters. Is it too long? Should I remove hyphens? Which is better?
On-Page Optimization | | tiffany11030 -
How do you handle URLs with slashes?
I asked this question before, but with a different scenario. I upgraded my plan to a more advanced cart and all of my URLs changed about 1.5 years ago. I knew nothing about redirects and such, so none of that was done. Basically, let's say my site was: http://www.abc.com, but when people actually visit my site, they are directed to https://www.abc.com/. I have asked my host about redirecting and she that it is not possible. In the past, the link shared has been just www.abc.com . Will this hurt my ranking? My second question is ...let's say I have a link http://www.abc.com/blog , but now, the link is http://www.abc.com/blog/ . Will I be affected, since all my old links omit the slash?
On-Page Optimization | | tiffany11030 -
Duplication in landing page
This is driving me mad, I have a site that for some reason google and moz pick up the landing page as a duplicate. They see "mysite/" and "mysite/index.html" as two different pages and giving me warnings for duplication. I have no 301 included at this time and I am using foundation as the base. This is occurring both on a localhost test bed and live....... anyone got an idea how to correct.
On-Page Optimization | | AndyBirtles0 -
URL Keyword Variations?
I'm aware that keywords in the url aren't as effective as they used to be, but I'm still convinced that they do have a significant impact (based on results in one of the niches I'm in). My question is, will variations of keywords and "hidden" keywords have as much value as an exact keyword? For example, let's say that I'm trying to target the keyword "day." Will including variations like "daily" in the url work just as well? What about a brand name that includes the keyword hidden in its name, like "Dayest"? And, as a followup question, does including "stop" words have any effect? For example, if I'm trying to target the keyword "Day of the Month", would including "day" and "month" in the url be just as effective as including "day of the month"?
On-Page Optimization | | JABacchetta0 -
What's the point of my blog?
My website, www.toplinecomms.com has a reasonably good blog that gets quite good interaction and sharing. I introduced the blog at the start of 2013 because the general sentiment from all the SEO books and articles I had read was that a good blog could be invaluable to a search marketing campaign. The posts on the blog are keyword optimised and they get great shares and social engagement. However, I have noticed that the blog is stealing my services' pages' thunder! There are some keywords that I am keen for our services pages to rank for, but the blog is beating them to it! So my question is: How should I be using my blog to get my services pages to rank higher?
On-Page Optimization | | HeatherBakerTopLine0 -
Duplicate Content- Best Practise Usage of the canonical url
Canonical urls stop self competition - from duplicate content. So instead of a 2 pages with a rank of 5 out of 10, it is one page with a rank of 7 out of 10.
On-Page Optimization | | WMA
However what disadvantages come from using canonical urls. For example am I excluding some products like green widet, blue widget. I have a customer with 2 e-commerce websites(selling different manufacturers of a type jewellery). Both websites have massive duplicate content issues.
It is a hosted CMS system with very little SEO functionality, no plugins etc. The crawling report- comes back with 1000 of pages that are duplicates. It seems that almost every page on the website has a duplicate partner or more. The problem starts in that they have 2 categorys for each product type, instead of one category for each product type.
A wholesale category and a small pack category. So I have considered using a canonical url or de-optimizing the small pack category as I believe it receives less traffic than the whole category. On the original website I tried de- optimizing one of the pages that gets less traffic. I did this by changing the order of the meta title(keyword at the back, not front- by using small to start of with). I also removed content from the page. This helped a bit. Or I was thinking about just using a canonical url on the page that gets less traffic.
However what are the implications of this? What happens if some one searches for "small packs" of the product- will this no longer be indexed as a page. The next problem I have is the other 1000s of pages that are showing as duplicates. These are all the different products within the categories. The CMS does not have a front office that allows for canonical urls to be inserted. Instead it would have to be done going into the html of the pages. This would take ages. Another issue is that these product pages are not actually duplicate, but I think it is because they have such little content- that the rodger(seo moz crawler, and probably googles one too) cant tell the difference.
Also even if I did use the canonical url - what happened if people searched for the product by attributes(the variations of each product type)- like blue widget, black widget, brown widget. Would these all be excluded from Googles index.
On the one hand I want to get rid of the duplicate content, but I also want to have these pages included in the search. Perhaps I am taking too idealistic approach- trying to optimize a website for too many keywords. Should I just focus on the category keywords, and forget about product variations. Perhaps I look into Google Analytics, to determine the top landing pages, and which ones should be applied with a canonical. Also this website(hosted CMS) seems to have more duplicate content issues than I have seen with other e-commerce sites that I have applied SEO MOZ to On final related question. The first website has 2 landing pages- I think this is a techical issue. For example www.test.com and www.test.com/index. I realise I should use a canonical url on the page that gets less traffic. How do I determine this? (or should I just use the SEO MOZ Page rank tool?)0 -
Duplicate content
crawler shows following links as duplicate http://www.mysite.com http://mysite.com http://www.mysite.com/ http://mysite.com. http://mysite.com/index.html How can i solve this issue?
On-Page Optimization | | bhanu22170