Duplicate Content Resolution Suggestion?
-
SEOmoz tools is saying there is duplicate content for:
What would be the best way to resolve this "error"?
-
Does having the line:
DirectoryIndex index.html
Have any use in addition to the lines you posted?
Thanks.
-
Stephen I agree with the KISS method. Using an htaccess RewriteCond is not the simplest solution for someone who does not know htaccess syntax. In an effort to fully answer this, here is the typical code we are referring to:
Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^yoursite.com
RewriteRule (.*) http://www.yoursite.com/$1 [R=301,L]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index.html\ HTTP/
RewriteRule ^index.html$ http://www.yoursite.com/ [R=301,L]The first 2 lines are typical commands to Follow Symbolic links, and make sure that the rewrite engine is in the on state.
The first RewriteCond looks at the host, and if it is not the www. version the RewriteRule will redirect the visitor to the http://www. version of your site.
The second RewriteCond looks at whether this is an index.html file, if it is, it will the RewriteRule will 301 redirect them to the version with out the index.html, just yoursite.com/
-
Im a KISS guy, duplicate content pages should just be handled with a rewrite, then they don't appear in any of your stats, attract links, spread your like/tweet numbers over multiple pages, if you are using xml files to keep tabs on your indexing etc and give you a better idea of whats going on on your site
Also, you have to take into account Facebook likes, page tweets, +1s etc - does rel canonical work on the social graph data?
rel canonical sort orders etc but if its a pure duplicate, then 301
"Dont link to page X on your site" isnt really a good solution in my eyes, too much room for error
-
Completely agree.
I think II may have been slightly confused by thinking the default for www.mydomain.com/ was not iindex.html
-
Yes, by the original posting your impression is correct and are the same page, but you can't 301 an index.html page to the domain where the index.html is the page that shows by default.
You could use an htaccess RewriteCond, but could be a little overkill for this situation, where adding a canonical will solve it.
-
I was under the impression that www.mydomain.com/ and www.mydomain.com/index.html were both indexing but are the same page
-
If the index.html is their home page that shows up when just doing the domain: http://www.mydomain.com/
then what would you 301 to? Are you assuming that it is a site that is using a index.php, index.htm, index.asp, etc.?
PlasticCards stated that there is a duplicate content, therefore the index.html page actually exists and should use a canonical.
-
I think in that particular situation I would use 301 as there really isn't a separate use for the /index.html page
-
or 301 redirect in your .htaccess and then you don't have to worry about link issues etc
-
Use a rel='canonical' and use the non index.html for the href;
also don't link to the index.html from anywhere.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to handle city-based product selection and duplicate content?
Hi everyone, I've been searching the interwebs for a solution to my problem, but haven't really found anything conclusive. I've got a client with duplicate content issues; their website not only has a nation-wide website, but also 10 different sub-categories for different cities, with each subcategory having the same content as the main website. The reason they wanted city-based websites was due to the changing product offerings in each city. So City 1 may not have all the products available that City 2 does. Needless to say this has caused some duplicate content issues as most sections of the website have been multiplied by 10. When a visitor lands on any page of the website, they are greeted by a pop up asking for their location, which will then redirect them to their selected version of the website. As the copy cannot really be changed enough for each city to make it unique, I've been looking into canonical tags, but this would mean the localised versions will not be indexed by Google. Has anyone had any experience of a similar situation where the product range changes according to location, but it doesn't hurt SEO? Thanks in advance for any advice!
Technical SEO | | Nimbus30000 -
301 duplicate content dynamic url
I have a number of pages that appear as duplicate titles in google webmaster. They all have to do with a brand name query. I want to 301 these pages since I'm going to relaunch my new website on wordpress and don't want to have 404s on these pages. a simple 301 redirect doesn't work since they are dynamic urls. here is an example: /kidsfashionnetherlands/mimpi.html?q=brand%3Amim+pi%3A&page=2&sort=relevance /kidsfashionnetherlands/mimpi.html?q=mim+pi&page=3&sort=relevance /kidsfashionnetherlands/mimpi.html?q=mim+pi&page=5&sort=relevance should all be 301 to the original page that I want to remain indexed: /kidsfashionnetherlands/mimpi.html I have a lot of these but for different queries. Should I do a 301 on each of them to avoid having 404s when I change my site to wordpress? Thanks
Technical SEO | | dashinfashion0 -
Duplicate Content Question (E-Commerce Site)
Hi All, I have a page that ranks well for the keyword “refurbished Xbox 360”. The ranking page is an eCommerce product details page for a particular XBOX 360 system that we do not currently have in stock (currently, we do not remove a product details page from the website, even if it sells out – as we bring similar items into inventory, e.g. more XBOX 360s, new additional pages are created for them). Long story short, given this way of doing things, we have now accumulated 79 “refurbished XBOX 360” product details pages across the website that currently, or at some point in time, reflected some version of a refurbished XBOX 360 in our inventory. From an SEO standpoint, it’s clear that we have a serious duplicate content problem with all of these nearly identical XBOX 360 product pages. Management is beginning to question why our latest, in-stock, XBOX 360 product pages aren't ranking and why this stale, out-of-stock, XBOX 360 product page still is. We are in obvious need of a better process for retiring old, irrelevant (product) content and eliminating duplicate content, but the question remains, how exactly is Google choosing to rank this one versus the others since they are primarily duplicate pages? Has Google simply determined this one to be the original? What would be the best practice approach to solving a problem like this from an SEO standpoint – 301 redirect all out of stock pages to in stock pages, remove the irrelevant page? Any thoughts or recommendations would be greatly appreciated. Justin
Technical SEO | | JustinGeeks0 -
Duplicate page content
Hello, The pro dashboard crawler bot thing that you get here reports the mydomain.com and mydomain.com/index.htm as duplicate pages. Is this a problem? If so how do I fix it? Thanks Ian
Technical SEO | | jwdl0 -
Press Releases & Duplicate Content
How do you do press releases without duplicating the content? I need to post it on my website along with having it on PR websites. But isn't that considered bad for SEO since it's duplicate content?
Technical SEO | | MercyCollege0 -
Is 100% duplicate content always duplicate?
Bit of a strange question here that would be keen on getting the opinions of others on. Let's say we have a web page which is 1000 lines line, pulling content from 5 websites (the content itself is duplicate, say rss headlines, for example). Obviously any content on it's own will be viewed by Google as being duplicate and so will suffer for it. However, given one of the ways duplicate content is considered is a page being x% the same as another page, be it your own site or someone elses. In the case of our duplicate page, while 100% of the content is duplicate, the page is no more than 20% identical to another page so would it technically be picked up as duplicate. Hope that makes sense? My reason for asking is I want to pull latest tweets, news and rss from leading sites onto a site I am developing. Obviously the site will have it's own content too but also want to pull in external.
Technical SEO | | Grumpy_Carl0 -
Strange duplicate content issue
Hi there, SEOmoz crawler has identified a set of duplicate content that we are struggling to resolve. For example, the crawler picked up that this page www. creative - choices.co.uk/industry-insight/article/Advice-for-a-freelance-career is a duplicate of this page www. creative - choices.co.uk/develop-your-career/article/Advice-for-a-freelance-career. The latter page's content is the original and can be found in the CMS admin area whilst the former page is the duplicate and has no entry in the CMS. So we don't know where to begin if the "duplicate" page doesn't exist in the CMS. The crawler states that this page www. creative-choices.co.uk/industry-insight/inside/creative-writing is the referrer page. Looking at it, only the original page's link is showing on the referrer page, so how did the crawler get to the duplicate page?
Technical SEO | | CreativeChoices0 -
WordPress Duplicate Content Issues
Everyone knows that WordPress has some duplicate content issues with tags, archive pages, category pages etc... My question is, how do you handle these issues? Is the smart strategy to use robots meta and add no follow/ no index category pages, archive pages tag pages etc? By doing this are you missing out on the additional internal links to your important pages from you category pages and tag pages? I hope this makes sense. Regards, Bill
Technical SEO | | wparlaman0