Duplicate content reported for totally different pages
-
Hi,
The Moz report is showing just over 21,500 duplicate page issues on our site. This is more or less every page we have. However when I look at the pages it says are duplicates they are totally different (it could for example report that a news page for 2009 is the same as a product page just added which has no relation when you read the content or view the page).
What sort of thing could it be picking up as duplicate content? I assume it must be something in the HTML for the site rather than the actual page content as there is no cross over at all on the pages highlighted. The only issue I can currently identify is that the menu for the mobile version of the site has a huge number of internal links which I will cut down. If the tools purely look at HTML content this could be seen as duplicate but shouldn't it be clever enough to realise what is content and what is site structure?
Thanks,
-
Hi Stephen!
Thanks for writing with a great question! Campaigns have up to a 90% tolerance for duplicate content. This includes all the source code on the page and not just the viewable text. So if a URL is at least up to 90% similar in code to another URL, this warning will appear. Although the pages in question are may appear to be different on the front end, they are actually duplicates based on this percentage.
We don't know what standard Google uses, but it's safe to say they are a bit more sophisticated than us - so you might be okay in this regard as long as you have a couple hundred words of unique text per page. Google won't say how much duplicate content is too much, so we like to be better safe than sorry.
Hope this helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz Crawler fails on the first page
Hi guys, Can anybody shed some light, I'm running a crawl on a client's website but it's failing on the homepage and not crawling any other pages. It appears to be throwing an 804: HTTPS (SSL) error and then terminating the crawl. Now, the page in question was serving up mixed content up until about 4 days ago, but has since been fixed. I read that we should wait at least 48 hours before initiating another crawl to avoid hitting a cached version - which I did, but it still appears to be having issues. Is there anything specific I can do to get around this issue? I'm on a trial account and this feature is one I'm keen to test, so there is a bit of a time constraint. Any help is greatly appreciated! Thanks in advance!
Moz Bar | | philipdanielhayton0 -
On Page Grader not working on the fly
On Page Grader has this lovely little re-grade page button next to the URL and term field but it's never worked on the fly for me. What's going on here? You discover something critical, fix it and you want the satisfaction of re-grading to test it and get that A but it doesn't work. I can see the changes on the site so why not??
Moz Bar | | wearehappymedia0 -
I'm getting a Crawl error 605 Page Banned by robots.txt, X-Robots-Tag HTTP Header, or Meta Robots Tag
The website is www.bigbluem.com and is a wordpress site. I'm getting the following error: 605 Page Banned by robots.txt, X-Robots-Tag HTTP Header, or Meta Robots Tag But what is weird is the domain it lists below that is http://None/BigBlueM.com Any advice?
Moz Bar | | TumbleweedPDX1 -
Rank Tracker - URLs are Different when Exporting to CSV
When exporting to CSV in Rank Tracker, many of the URLs are reduced to the root domain instead of the full, ranking URL as seen within the tool. Right now the URLs must be copied/pasted or manually edited afterwards in the CSV. However, it doesn't happen to every item. A few of them do show the correct URL after being exported. Any idea if this is a bug or just an odd thing the export does?
Moz Bar | | AlfredGoldberg1 -
Error for a page that doesn't exist.
Hi, I'm just trailing this service, and I have a couple of questions that I hope someone can help with. 1. I am getting a high priority error regarding a page not being able to be crawled - a 4XX error. Problem is, there is no such page in existence. The URL is my site/comments/feed It's driving me crazy. 2. I'm also getting errors based on missing meta tags in blog posts. I am adding tags at the time of posting, so I am unsure why these errors are showing up. Actually, I didn't add tags to all posts - but there are errors on ALL posts, even those I added tags to. Any help would be wonderful. Thanks!!! Hugh
Moz Bar | | hughanderson0 -
Keyword Difficulty and SERP Analysis Full report not working?
It's been a few days now that I am having difficulties getting a Full Report on certain keywords in the "Keyword Difficulty" tool. It's been stuck on "pending..." for over 12 hours. I also noticed that some keywords are returning zero exact volume when I know 100% that it's returning 5K+ in GKP. Anyone else having the same issues?
Moz Bar | | mediadispaysolutions0 -
Why my on-page report card won't improve?
For my page www.homedestination.com/Minneapolis-homes-for-sale.html, Moz ranks the page as an "F" for the keyword "Minneapolis home buyers". All data in the report card is from months ago. using the "Grade my On-Page Optimization" button fails to show any updates - even after months. For example the title is actually "Minneapolis Homes For Sale| buying a Minneapolis home | Homebuyers" while Moz says it is ""Minneapolis Homes For Sale | Home Destination | Jenna Thuening" Moz says the description tag is "Realtor Jenna Theuning helping real estate buyers find Minnetonka homes for sale, owner of Home Destination. CDPE also helps with foreclosure and short sales buying." and "Minneapolis homes for sale, and Minneapolis listing, and Minneapolis upscale real estate, and Minneapolis lakeside properties provided by Home Destination"" when it is actually "Minneapolis homes for sale, and Minneapolis listing, Minneapolis home buyers, and Minneapolis lakeside properties provided by Home Destination." Additionally, my weekly ranking reports seems "stuck" and continues to show many "F" ranked keywords that have been updated for some time. Amy advice on how to see improvements?
Moz Bar | | jessential1 -
404 Crawl Diagnostics Report MOZ
Hi, I keep getting 404's appear in the Crawl Diagnostic error warnings. How do I find out which pages are linking to these 404 pages? How is MOZ finding them? thanks Ben
Moz Bar | | bjs20100