Crawl reports urls with duplicate content but its not the case
-
Hi guys!
Some hours ago I received my crawl report.I noticed several records with urls with duplicate content so I went to open those urls one by one.
Not one of those urls were really with duplicate content but I have a concern because website is about product showcase and many articles are just images with href behind them. Many of those articles are using the same images so maybe thats why the seomoz crawler duplicate content flag is raised. I wonder if Google has problem with that too.See for yourself how it looks like:
http://by.vg/NJ97y
http://by.vg/BQypEThose two url's are flagged as duplicates...please mind the language(Greek) and try to focus on the urls and content.
ps: my example is simplified just for the purpose of my question.
<colgroup><col width="3436"></colgroup>
| URLs with Duplicate Page Content (up to 5) | -
Disclaimer: I just answered a question just like this on another thread, so I literally copied and pasted my response from there, and edited where necessary.
The SEOmoz web app uses a similarity threshold of 95% of the html code. This takes everything on the page, both hidden and visible into account.
In this case, it's counting all of the navigation and sidebar as well, which is significant. What's left of the unique content - the part that matters, makes up less than 5% of the code.Here's a tool you can use to check the similarity: http://www.duplicatecontent.net/
I ran the pages through a couple of tools which showed 98% similarity. (but only 75% text similarity, which is good, but not great)
SEOKeith is absolutely right that there's very little on those pages to help them rank. Without text, you're fighting an uphill battle.
Hope this helps! Best of luck with your SEO.
-
Yeah, thats what I m going to do in my next meeting. Either way I also feel such websites need to have more pics than anything else, maybe a blog page or separate pages with articles could link to those products one by one with related description having a side content website for the actual product pages.
-
Maybe explain to the client it's not going to rank as well without text and has less chance of getting found by searches (generally speaking...).
I get duplicate content flagging as well sometimes, I check the pages manually when it happens.
-
Thanks Keith. I ve been using seomoz for some days so I wasnt sure about this.
Client wants website with as less text as possible so I guess my only hopes are title and alt attributes.
-
Those pages are very similar so it's probably throwing the duplicate content switch in SEOmoz, you might want to ignore it in this case.
I would add some more text to those pages personally to aid with ranking, you can position the text over the images with CSS.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why is MOZ crawl is returning URLs with variable results showing Missing Meta Desc? Example: http://nw-naturals.net/?page_number_0=47
Can you help me dive down into my website guts to find out why the MOZ crawl is returning URLs with variable results? And saying this is missing a description when it's not really a page? Example: http://nw-naturals.net/?page_number_0=47. I've asked MOZ but it's a web development issue so they can't help me with it. Has anyone had an issue with this on their website? Thank you!
Moz Pro | | lewisdesign0 -
Duplicate Content & Rel Canonical Tag not working
I'm really questioning the legitimacy of the duplicate content flags with moz. I'm building a website that sells home decor products and a lot of the pages are similar in structure (As would be expected with a store that sells thousands of individual products). It seems a little overkill to me to flag the following pages as duplicate content. They have different urls, titles, h1, h2, and h3 tages, different meta tags, etc. Right now, it's saying that the following have duplicate page content: http://www.countryporchhomedecor.com
Moz Pro | | cp_web
http://countryporchhomedecor.com/park-designs/pillows/christmas-vacation-embroidered-pillow
http://www.countryporchhomedecor.com/donna-sharp/throws/camo-bear-throw
http://countryporchhomedecor.com/park-designs/teapots/wonderland-teapot
http://countryporchhomedecor.com/park-designs/rag-rugs/cambridge-rug-36x60
http://www.countryporchhomedecor.com/donna-sharp/lodge-quilts/king%2C-woodland
http://www.countryporchhomedecor.com/park-designs/rag-rugs/redmon-rag-rug-36x60
http://www.countryporchhomedecor.com/park-designs/valances/hearthside-valance-72x14
http://countryporchhomedecor.com/park-designs/valances/hearthside-valance-72x14
http://countryporchhomedecor.com/donna-sharp/lodge-quilts/king,-woodland
http://www.countryporchhomedecor.com/park-designs/teapots/wonderland-teapot
http://countryporchhomedecor.com/donna-sharp/throws/camo-bear-throw
http://countryporchhomedecor.com/park-designs/accessories/home-place-tumbler
http://www.countryporchhomedecor.com/donna-sharp/lodge-quilts/king,-woodland
http://www.countryporchhomedecor.com/park-designs/rag-rugs/cambridge-rug-36x60
http://www.countryporchhomedecor.com/park-designs/pillows/christmas-vacation-embroidered-pillow
http://www.countryporchhomedecor.com/donna-sharp/lodge-quilts/king%2C-woodland?pi=18
http://countryporchhomedecor.com/donna-sharp/lodge-quilts/king%2C-woodland
http://www.countryporchhomedecor.com/park-designs/accessories/home-place-tumbler
http://countryporchhomedecor.com/park-designs/rag-rugs/redmon-rag-rug-36x6 Any ideas? Also, it seems like it's not honoring the rel-canonical tag. It keeps saying that pages with a rel canonical tag are duplicates when some of the urls that it's flagging shouldn't even be indexed because of the canonical tag. The "pi" in the query string should not be indexed! http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=3
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=6
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=7
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=6
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=10
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=8
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=8
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=7
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=7
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=1
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=8
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=5
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=10
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=3
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=5
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=4
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=9
http://www.countryporchhomedecor.com/bedding-%26-quilts/shams/standard-shams
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=1
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=6
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=1
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=5
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=2
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=9
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=4
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=3
http://countryporchhomedecor.com/bedding-%26-quilts/shams/standard-shams
http://www.countryporchhomedecor.com/bedding-%26-quilts/shams/standard-shams?pi=18
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=9
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=10
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?pi=18&page=2
http://countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=2
http://www.countryporchhomedecor.com/bedding-&-quilts/shams/standard-shams?page=40 -
Block Moz (or any other robot) from crawling pages with specific URLs
Hello! Moz reports that my site has around 380 duplicate page content. Most of them come from dynamic generated URLs that have some specific parameters. I have sorted this out for Google in webmaster tools (the new Google Search Console) by blocking the pages with these parameters. However, Moz is still reporting the same amount of duplicate content pages and, to stop it, I know I must use robots.txt. The trick is that, I don't want to block every page, but just the pages with specific parameters. I want to do this because among these 380 pages there are some other pages with no parameters (or different parameters) that I need to take care of. Basically, I need to clean this list to be able to use the feature properly in the future. I have read through Moz forums and found a few topics related to this, but there is no clear answer on how to block only pages with specific URLs. Therefore, I have done my research and come up with these lines for robots.txt: User-agent: dotbot
Moz Pro | | Blacktie
Disallow: /*numberOfStars=0 User-agent: rogerbot
Disallow: /*numberOfStars=0 My questions: 1. Are the above lines correct and would block Moz (dotbot and rogerbot) from crawling only pages that have numberOfStars=0 parameter in their URLs, leaving other pages intact? 2. Do I need to have an empty line between the two groups? (I mean between "Disallow: /*numberOfStars=0" and "User-agent: rogerbot")? (or does it even matter?) I think this would help many people as there is no clear answer on how to block crawling only pages with specific URLs. Moreover, this should be valid for any robot out there. Thank you for your help!0 -
Duplicate Page Content on pages that appear to be different?
Hi Everyone! My name's Ross, and I work at CHARGED.fm. I worked with Luke, who has asked quite a few questions here, but he has since moved on to a new adventure. So I am trying to step into his role. I am very much a beginner in SEO, so I'm trying to learn a lot of this on the fly, and bear with me if this is something simple. In our latest MOZ Crawl, over 28K high priority issues were detected, and they are all Duplicate Page Content issues. However, when looking at the issues laid out, the examples that it gives for "Duplicate URLs" under each individual issue appear to be completely different pages. They have different page titles, different descriptions, etc. Here's an example. For "LPGA Tickets", it is giving 19 Duplicate URLs. Here are a couple it lists when you expand those:
Moz Pro | | keL.A.xT.o
http://www.charged.fm/one-thousand-one-nights-tickets
http://www.charged.fm/trash-inferno-tickets
http://www.charged.fm/mylan-wtt-smash-hits-tickets
http://www.charged.fm/mickey-thomas-tickets Internally, one reason we thought this might be happening is that even though the pages themselves are different, the structure is completely similar, especially if there are no events listed or if there isn't any content in the News/About sections. We are going to try and noindex pages that don't have events/new content on them as a temporary fix, but is there possibly a different underlying issue somewhere that would cause all of these duplicate page content issues to begin appearing? Any help would be greatly appreciated!0 -
Website Issues - Duplicate Content
Hello, I'm fairly new to using Moz and I logged on this morning to find Issues have been found in one of the websites - 22 High Priority and 44 Medium. I know it's due to duplicate content in the blog, but i can't figure out what is duplicated? I've only recently come on board this website so I don't know if the content has been plagiarised or what? The link to the site is here: delacyspa.co.uk Any help would be appreciated. Thanks zFxQmmd
Moz Pro | | Cowbang0 -
URL Encoding
HI SEOmoz has finished crawling the site and surprised me with nearly 4k of 301's all the 301 are on my deal pages Example of the 301 http://www.economy-car-leasing.co.uk/van-leasing-deals/ford/transit-lease/transit-lwb-el-minibus-diesel-rwd-high-roof-17-seater-tdci-135ps%3D586165 as you can see from the above URL it returns a 404 but the URL is actually sent as below http://www.economy-car-leasing.co.uk/van-leasing-deals/ford/transit-lease/transit-lwb-el-minibus-diesel-rwd-high-roof-17-seater-tdci-135ps=586165 For some reason SEOmoz crawler is converting the = to %3d and reporting its a 301 even though it returns 404 Is this an error on SEOMOZ part ? or is there an error on my site Googlebot when i do a fetch as Google bot returns all on with the = sign and every other tool i have tried is ok too so not sure why SEOMOZ is seeing it different and then adding the URL as a 301 I am hoping this is just a glitch on the report tool part as im struggling since a recent site 301
Moz Pro | | kellymandingo0 -
My Campaign has been crawling for about a week now
Can anyone tell me why one of my campaigns has been stuck in crawl mode for about a full week and it is still not done?!?!
Moz Pro | | nazmiyal0 -
All seo reports
I want to find out what all the report's correct definition and when I see improving or going down what does it mean for eg. Organic search report? Keywords? or non-paid keywords?
Moz Pro | | ITWEBTEAM0