Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

API for testing duplicate content

Technical SEO

1133

Sebes last edited by

Does anyone know a service or API or php lib to compare two (or more) pages and to return their similiarity (Level-3-Shingles).

API would be greatly prefered.
1 Reply Last reply
Reply Quote 0
Sebes @EricaMcGillivray last edited by

Hey Erica,

thanks for your answer. What I need is a way to decide on-the-fly whether two pages are similar or not. If they are too similar I need to depublish or at least rel canonical one of those.

Best solution would be an API that takes 2 pages, but it seems as if I have to build it myself then.

Thanks for your efforts.
1 Reply Last reply
Reply Quote 0
EricaMcGillivray last edited by

While I don't know of an API that does that, you can set up your site using the SEOmoz tools and our Crawl Diagnostics section does look for Duplicate Cotent.
1 Reply Last reply
Reply Quote 0

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Duplicate content

I have one client with two domains, identical products to appear on both domains. How should I handle this?
Technical SEO | | Hazel_Key

0
What is the Impact of Duplicate Content on Multiple Managed Property Domains?

Hi Moz Community! Our team is having an internal (and external) debate regarding the extent and implications of duplicate content for a hospitality client that I would love to get some feedback on. I unfortunately cannot divulge the brand/URL, but will give as much info as possible. The brand in question manages dozens of properties in the US and worldwide and has recently rolled up all of the domains under a singular brand.com domain. So whereas the properties used to have their own domains (property1.com, property2.com, etc...), they are now housed in sub-folders (brand.com/property1, brand.com/property2.com and so forth). The concern we have is that they launched the new brand site with all of the property sites/content rolled up under the new brand.com domain, however all of the individual property sites and their pages are still live as well. All of the canonicals on both brand.com as well as property1.com (property2.com, property3.com, etc...) are self-referencing (so the canonicals for brand.com/property1 and all of its sub-ages do not point to the still live property1.com and all of its sub-pages, for example). On the brand side, they believe this is the best path forward as brand.com grows and gains some authority, with the later intent on eventually redirecting the individual property domains - but we are unclear of that timeline (though we do think its more months as opposed to days/weeks) So our questions for the community here are: What is the perceived impact in this state of limbo to the individual property sites (ideally they house the original content and have the history, but could Google still give preference to the brand.com/property URLs and/or could both of them suffer in rank/search experience from the duplicate content an non-uniform presentation?) Could brand.com be "dinged" so-to-speak due to launching with this much duplicate content? (And if so, could that affect how quickly normalization occurs after the property sites are finally redirected?) Anything else we should consider/Any other feedback from the community? Thank you all for your time and support!
Technical SEO | | imiJoe

0
Duplicate Content?

My site has been archiving our newsletters since 2001. It's been helpful because our site visitors can search a database for ideas from those newsletters. (There are hundreds of pages with similar titles: archive1-Jan2000, archive2-feb2000, archive3-mar2000, etc.) But, I see they are being marked as "similar content." Even though the actual page content is not the same. Could this adversely affect SEO? And if so, how can I correct it? Would a separate folder of archived pages with a "nofollow robot" solve this issue? And would my site visitors still be able to search within the site with a nofollow robot?
Technical SEO | | sakeith

0
Duplicate content in Magento

Hi all We got some serious issues with duplicate content on a Magento site that we are marketing. For example: http://www.citcop.se/varmepumpar-luft-luft/panasonic/panasonic-nordic-ce9nke-5-0kw http://www.citcop.se/panasonic/panasonic-nordic-ce9nke-5-0kw http://www.citcop.se/panasonic-nordic-ce9nke-5-0kw All of the above seem to work just fine as it is now but since they are excatly the same product they should ofcourse do a 301 redirect to the main page. Any ideas on how to sort this out in Magnto without having to resort to manual work in .htaccess? Have a great day Fredrik
Technical SEO | | Resultify

0
Similar Content vs Duplicate Content

We have articles written for how to setup pop3 and imap. The topics are technically different but the settings within those are very similar and thus the inital content was similar. SEOMoz reports these pages as duplicate content. It's not optimal for our users to have them merged into one page. What is the best way to handle similar content, while not getting tagged for duplicate content?
Technical SEO | | Izoox

0
Google Duplicate Content Penalty On My Own Site?

I am certain that I have hit a google penalty filter for my site http://www.playpokeronline.ca for my main keywords "play poker online" in google.ca I rank 670th and used to be on the first page between 1 and 10 in June. On Bing I am like 9th On my site I found the entire site duplicated as follows Original: www.playpokeronline.ca Duplicate www.playpokeronline.ca/playpokeronline/ this duplicate was not intentional and seems to be a result of my hosting at godaddy. for every page on my site and it shows up in webmaster tools I blocked the duplicate with robots.txt and a few days ago dropped it and wrote a rel=connonical tag in the top of each page visitors dropped from 100 per day in august to 12-20 in the last month. Google says that if duplicate content is made to try to game serps they may filter or penalize my site. Have I triggered this penalty or a different sort of over optimization penalty? Will the rel= canonical tags fix this or should i do something else? This Penalty Business is Not my Idea of a good time Thank You Jeb
Technical SEO | | PokerCanada

0
Duplicate Page Content

Hi within my campaigns i get an error "crawl errors found" that says duplicate page content found, it finds the same content on the home pages below. Are these seen as two different pages? And how can i correct these errors as they are just one page? http://poolstar.net/ http://poolstar.net/Home_Page.php
Technical SEO | | RouteAccounts

0
About duplicate content

Hi i'm a new guy around here, but i'm having this problem in my website. Using de Seomoz tools i ran a camping to my website, in results i get to many errors for duplicate conten, for example, http://www.mysite/blue/ http://www.mysite/blue/index.html, so my question is, what is the best way to resolve this problem, use a 301 or use the rel canonical tag? Wich url will be consider for main url, Thanks for yor help.
Technical SEO | | NorbertoMM

0