Duplicate Page Titles and Content
-
The SeoMoz crawler has found many pages like this on my site with /?Letter=Letter, e.g. http://www.johnsearles.com/metal-art-tiles/?D=A. I believe it is finding multiple caches of a page and identifying them as duplicates. Is there any way to screen out these multiple cache results?
-
I think I figured out what to add to Robots.txt to screen out any url with an '?' in it. I believe these ?urls are session IDs for Urls. I'll see what Roger-bot does next time it crawls my site.
Disallow: /*?
-
Hey John,
My apologies for any issues that you are experiencing with our service. I would definitely like to address any other issues, besides this one, that you may be experiencing. You could either respond to this Q&A thread or submit a private customer support ticket to our help team. If you go to our help hub (www.seomoz.org/help) you can easily submit a ticket by clicking the contact help team button.
As for your duplicate content question, it is important to know that any time the same content is found on more than one URL that it is considered duplicate content. WordPress is a good example where duplicate is often found but can be easily addressed.
In WordPress you could have your homepage www.domain.com and an author page www.domain.com/author/authorname. If your blog only has one author though this author page is going to be identical to your homepage and the result is your site having duplicate content. There are a few ways to resolve this though with the most popular being simply preventing access to the author page and redirecting it back to the homepage. This would prevent other sites from linking to these duplicate pages and they would instead link directly to the homepage.
Another option would be to use meta robots noindex and follow tags on the duplicate page, in this case the author page. This would prevent the page from being indexed but will still allow the links on the page to be found and crawled. You can also prevent access to these pages in your robots.txt file and our crawler can be isolated by using the user-agent rogerbot.
I hope that makes sense.
Let me know if you have any additional questions or concerns.
Kenny
-
Thanks Guy. I was thinking of subscribing to SeoMoz but the site reports have been less than useful. This is just one of 5 issues I've found.
-
So far no. Until they fix that little error you can use Google Webmaster Tool's to double check for real duplicate content.
The spider is seeing whatever.php?var=1 as a different page because some sites just use index.php?p=103 to be a page and p=102 another page. While others use the variables in the URL on the same page.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content in crawl despite canonical
Hi! I've had a bunch of duplicate content issues come up in a crawl, but a lot of them seem to have canonical tags implemented correctly. For example: http://www.alwayshobbies.com/brands/aztec-imports/-catg=Fireplaces http://www.alwayshobbies.com/brands/aztec-imports/-catg=Nursery http://www.alwayshobbies.com/brands/aztec-imports/-catg=Turntables http://www.alwayshobbies.com/brands/aztec-imports/-catg=Turntables?page=0 Aztec http://www.alwayshobbies.com/brands/aztec-imports/-catg=Turntables?page=1 Any ideas on what's happening here?
Moz Pro | | neooptic0 -
Only few pages (308 pages of 1000 something pages) have been crawled and diagnosed in 4 days, how many days till the entire website is crawled complete?
Setup campaign about 4-5 days ago and yesterday rogerbot said 308 pages were crawled and the diagnostics were provided. This website has over 1000+ pages and would like to know how long it would take for roger to crawl the entire website and provide diagnostics. Thanks!
Moz Pro | | TejaswiNaidu0 -
SEOmoz crawler and duplicate content
Does anybody know if the SEOmoz crawler picks up canonical tags when its looking for duplicate content? I've got a ton of errors in one of my projects even though they all have canonical tags. Thanks!
Moz Pro | | neooptic0 -
Duplicate Content
I have tried searching for an exact example of the issues I am seeing, but didn't come up with anything. I decided to post my own question so I can get a direct answer on what I am experiencing. I recently took over a website and its' existing SEO practices with it. Upon placing the site on SEOmoz, I received many (LOTS) of duplicate content warnings. Pretty much, this is how the website is setup: domain.com/keyword-is-here/ but it is also coming up as domain.com/keyword-is-here/index.htm - Should I setup a redirect so domain.com/keyword-is-here/index.htm points to domain.com/keyword-is-here.htm or should I just leave it alone since it's pointing to the same exact? Any information on this questions is greatly appreciated in advance.
Moz Pro | | EQ-Richie0 -
Why aren't canonical tags reducing duplicate page title/content?
We have canonical tags set up for a feature page on one of our sites. This site has an image gallery controlled by javascript. To aid the user experience the image can also be specified by a URL parameter (the javascript also uses this URL to fetch the images). The SEOMoz report complains that the links to these images have duplicate page titles and content. To try and combat this we set canonical tags to point only to the original page, without the slideshow parameter. e.g. http://www.example.com/feature-page/ http://www.example.com/feature-page/?slideshow=1 -> canonical tag set to http://www.example.com/feature-page/ http://www.example.com/feature-page/?slideshow=2 -> canonical tag set to http://www.example.com/feature-page/ The latest SEOMoz report has come back and the errors still exist. What can we do to remove these error messages? Thanks
Moz Pro | | TJSSEO1 -
Seomoz & Duplicate Page Content Issue?
Hi, What is the criteria on Seomoz Crawl Diagnostic Report? I got a long list of URLs indicating Content that is identical (or nearly identical) to content on other pages of your site forces your pages to unnecessarily compete with each other for rankings. But as I gone through none of the reported pages duplicate. What should I do? Thanks in Advance
Moz Pro | | VipinLouka780 -
Duplicate page error from SEOmoz
SEOmoz's Crawl Diagnostics is complaining about a duplicate page error. I'm trying to use a rel=canonical but maybe I'm not doing it right. This page is the original, definitive version of the content: https://www.borntosell.com/covered-call-newsletter/sent-2011-10-01 This page is an alias that points to it (each month the alias is changed to point to the then current issue): https://www.borntosell.com/covered-call-newsletter/latest-issue The alias page above contains this tag (which is also updated each month when a new issue comes out) in the section: Is that not correct? Is the https (vs http) messing something up? Thanks!
Moz Pro | | scanlin0