Checking for content duplication against content on your own site.
-
We are currently trying to rewrite our product descriptions and I'm afraid some of the salespeople that are writing the descriptions are plagiarizing one-another's writing. Is there a content duplication checker that will allow you to check a piece of writing against a specific site rather than all of the web?
-
I assume that you have an admin section in the CMS where you are editing and entering these articles before they go live.
You need to get a developer to simply write a search algo that when you create a new article and before it goes live, it takes sections of your content and looks for matches/duplicates. You can set a requirement that it has to match on a minimum of a 4 to 5 word string and other such limitations to make sure you are not matching too many items. It will take a few tests to find a sweet spot of too many matches vs not enough.
With 17K pages, this is the only way you can really do this in an efficient way, you need some IT support/development. They may have to create a reporting layer as well to help you sift through the results.
Good luck.
-
I have two dev servers, one of which it is possible to do what you're talking about but that is the absolute least efficient tool to use for this.
The crawl diagnostics are updated about once a week which means I would have to post the new content and hope I got it online in time for the crawl. If I didn't then I would have to wait an additional week to see results.
The crawl diagnostics also limits the amount of pages it will crawl on your site to 10,000. I stated before that I have over 17,000 pages. So even if I did use this method, the chances of that page being crawled is little better than 50/50.
Also, the crawl diagnostics only tell you what pages have duplicate content - not the exact content that was duplicated. That means I'd have to manually find the page I'm targeting, then follow the supposed duplicate content suggestions proposed by the crawler and find the similarities myself.
I think it's very safe to say that the crawl diagnostics, nor any product that SEOmoz provides, is an answer to my issue. If I thought it was, I would have already been using it and would not have posted this question.
-
Hi Michael,
Having a website that big means that you might have a test or dev environment.
If not create one.
if you have something like test.yourwebsite.com and submit it to the SEOmoz tools as a new project you can see a report before your website goes live.
Cornel
-
Those are good answers and would work on a smaller scale site. We currently have over 17,000 product pages so I can't really use either method. It's looking like a google custom search is the best bet even though I can't search an entire paragraph at a time.
-
Just off the top of my head, there are a few low tech ways to do it....
If you have Win 7 the searching has improved greatly - just move all files to a local machine - and search the directory you placed in for the content you are wanting to check - it will give all files that contain the words. (but can become overloading)
If you have dreamweaver or other enterprise level editor - almost all have a site search function to where you can search/profile code/text and have it find one by one which pages contain the searched terms - or globally list them.
Other than that, probably a custom script -or a google search for an HTML profiler might help?
Shane
-
That's for pages that are already published and crawled. I want to able to search my site for entire sentences and/or paragraphs of text that I have yet to publish so I can make sure it's not being used elsewhere on the site. The crawl diagnostics tell me I have duplicate content after the fact - I'm trying to take a proactive approach rather than reactive.
-
The duplicate content from you website is shown in the SEOmoz tools.
Check the Crawl Diagnostics Summary:
Cornel
-
That site searches the entire web for copies. I'm looking for something to crawl my own site for duplicate content.
-
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Two Day After Starting Moz Pro campaign i experienced Sudden Huge Traffic Drop and My site ranki Drops in my site..Please I need your help
Two Day After Starting Moz Pro campaign i experienced Sudden Huge Traffic Drop and Drops in my site..Please I need your help
On-Page Optimization | | zizutz0 -
Is this hidden content?
Hi all, I was wondering if the homepage of www.dirtylooks.com has hidden content in a search engines eyes. There is some text which appears underneath a tile called "hair tools" that has to be scrolled in order to be viewed by a visitor. As this isn't the typical white on white or off page by CSS hidden content are we in danger of being penalised?
On-Page Optimization | | BenfromBNKR0 -
Duplicated page titles
Dear friends, We have a problem which occurs on our page with duplicated page titles. Landlords are posting rooms on our page and most of them are giving the same name to the rooms and after that, we are getting more and more duplicated page titles. We are applying random whit this title tag: Accommodation for students in {city.name}: {name}. English title
On-Page Optimization | | Eurasmus.com
Certified student rooms in {city.name}: {name} English title
Erasmus room for students in {city.name} | {name} English title
Student room in {city.name}: {name} English title Also our title tag is sometimes to long but there is no possibility to make them shorter. I think. If anyone would have some idea be free to comment and help us. Kind regards Miško Macolić Tomičić0 -
Site restructure question
Our site was deigned years ago to target customers in specific cities, now we've grown beyond this and I believe it is time to change the site structure.
On-Page Optimization | | PM_Academy
Ignore the 302 from the root page. Current structure: (assuming you've never been to our site before) projectmanagementacademy.net 302->/select-location.php /select-location.php -> /city-name/pmp-training.php This page was meant to be a "homepage" for each city, pointless page really /city-name/pmp-training.php -> /ciy-name/product-name.php These pages are for each individual product My suggested site structure: /city-name/pmp-training.php becomes projectmanagementacademy.net no more redirect /city-name/pmp-training.php gets removed and 301 to root page. /product-name.php each product's page and you would select a location when necessary (some products are online only) would 301 each /city-name/product-name to corresponding product page /product-name/city-name.php could add these pages if we still wanted the city name in url for city specific products My thoughts here are /product-name.php would receive a higher % of link juice because there are fewer page between 2 vs 4 if you came to the root page. and 2 vs 3 if you came from the select-location page. Also instead of being split between over 50 locations, all these would be together on one page. Your thoughts? Would this change improve our SERP for those product pages? Would we see a drop off in traffic if we did this? How long, if done correctly, would it take to see the recovery of rankings and traffic? Could we 301 /select-location.php to the root page? Thanks in advance for your insights to this. Any answer is a good answer. Trenton0 -
Does schema.org assist with duplicate content concerns
The issue of duplicate content has been well documented and there are lots of articles suggesting to noindex archive pages in WordPress powered sites. Schema.org allows us to mark-up our content, including marking a components URL. So my question simply, is no-indexing archive (category/tag) pages still relevant when considering duplicate content? These pages are in essence a list of articles, which can be marked as an article or blog posting, with the url of the main article and all the other cool stuff the scheme gives us. Surely Google et al are smart enough to recognise these article listings as gateways to the main content, therefore removing duplicate content concerns. Of course, whether or not doing this is a good idea will be subjective and based on individual circumstances - I'm just interested in whether or not the search engines can handle this appropriately.
On-Page Optimization | | MarkCA0 -
Is this duplicate content okay?
We have a client who wants to rank locally, nationally and internationally for their products. I wrote a line that goes, "We can ship our products to you whether you’re here in Illinois, nationwide, or international." I added that line after a paragraph or two of unique product description on each of their 30-odd product pages. Will this damage their ranking? I tried researching this but only found full page duplicate content topics. Any advice would be great.
On-Page Optimization | | optimalwebinc0 -
Duplicate meta descriptions
Hi all, I'm using Yoast's SEO plugin and when I run a On Page report card here on SEOMOZ it says there are 2 descriptions tags I've been trying to fix this but can't (I'm new!) Anyone any ideas on this? Thanks Elaine
On-Page Optimization | | elaineryan0 -
How Do You Build Good Quality Content on an E-Commerce Site with over 1500 products?
I've been told for a while now that quality is king, and as far as the top level down goes, I have improved the quality of these pages incredibly, working from the landing page down to the categories. Now the problem comes when I know that if I describe each product dynamically with great content, this give the best result, yet doing this on over 1500 products is just a mammoth task for anyone, isn't it? I want a way to have good quality content on each of those pages without writing a million words.... is this possible?
On-Page Optimization | | frank-2443750