Duplicate Content for index.html
-
In the Crawl Diagnostics Summary, it says that I have two pages with duplicate content which are:
I read in a Dream Weaver tutorial that you should name your home page "index.html" and then you can let www.mywebsite.com automatically direct the user to index.html. Is this a bug in SEOMoz's crawler or is it a real problem with my site?
Thank you,
Dan
-
The code should definitely go into the websites root directory's .htaccess, however .htaccess can be weird, a few days ago I ran into a similar issue with a client's website, and I was able to remedy the issue with a variation of the code.
index Redirect RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)index.(php|html|htm|asp)\ HTTP/ RewriteRule ^(([^/]+/))index.(php|html|htm|asp)$ http://yoursite.com/$1 [R=301,L]
If you give me the URL for the site I will take a look at it and let you know what would be feasible.
-
Hi Daniel, can you share with us the URL of your site? We can take a look at it and give you a more precise answer that way. Thanks!
-
I eventually figured out that your method was a 301 redirect and I definitely broke my site trying to use the code you posted. .. haha. Its ok though. I just removed the code and it went back to normal. At first, I was editing the .htaccess file in the public_html folder which wasnt working. Then I tried the root folder for the site (I created the .htaccess file since it did not exist.) Neither of those worked. (I am using Bluehost so I do not think that I have root access and I am not sure if it is a Linux server or not.)
If there is an easy way to explain what I am doing wrong, please do so. Otherwise, I will use canonical.
Thanks for everything!
-
@Dan
Thanks for your reply. It seems like there are lots of different ways to solve this problem. I just watched this video on Matt Cutt's blog where he discusses his preference for 301 redirects over rel canonical tag.
Where would you say your solution fits in?
sorry about the delay of this response, i didn't realize the that you were asking me a question right away. When placing the code I provided in my previous answer this will cause a 301 perminant redirect to the original URL. That's actually what the
[R=301,L]
portion of the code is stating (R) redirect (301) status is referring to. After reviewing the Matt Cutts video, I realize that I should have asked you if you were operating on a Linux server that you had root access to. We actually utilize both redirects and canonical tags since it was recommended by the on-page optimization reports. Heck Google uses them, I would assume because it's easier for the user to be referred to a single page URL. Obviously though if you don't have server header access, and are not familiar with .htaccess (you can accidentally break your site) then the canonical solution is appropriate
-
Josh,
Thanks for your reply. It seems like there are lots of different ways to solve this problem. I just watched this video on Matt Cutt's blog where he discusses his preference for 301 redirects over rel canonical tag.
Where would you say your solution fits in?
Thanks,
Dan -
use the link rel tag for all my homepages for the http://www.yoursite.com
-
Odd enough I just recently answered this question. The SEOmoz crawler is correct, because without a redirect you will be able to access both versions of the page in your browser.
To resolve this issue simply rewrite the index.html to the root url by placing the following code into your .htaccess file into your root directory.
Options +FollowSymlinks RewriteEngine on
Index Rewrite RewriteRule ^index.(htm|html|php) http://www.yoursite.com/ [R=301,L] RewriteRule ^(.*)/index.(htm|html|php) http://www.yoursite.com/$1/ [R=301,L]
You can also do the same with the index file in any subdirectories that you might create, by simply placing a .htaccess into those sub directories and using variations of the above code. This is how you create nice tight URLs without the duplicate content issue that look like - http://www.semclix.com/design/business/
-
It is a problem which you need to fix. You need to canonicalize your pages.
Those are all various URLs which most likely lead to the same web page. I say "most likely" because these URLs can actually lead to different pages.
You need to tell crawlers and search engines how you organize your site. There are several ways to achieve canonicalization. The method I prefer is to add the following line of code to each page:
The URL provided should be the preferred URL for your page.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is switching from a very old HTML table site to HTML5 going to make a big difference
Hello, My site owner has been having calls that our old HTML site needs to switch to HTML5. Is it really worth the ubgrade from an archaic HTML site? Please explain. Bob
Web Design | | BobGW0 -
Duplicate items across different pages?
On our new website we have a testimonials page which you can cycle through them. We also have the testimonial on the our work / project page. Essentially this is duplicate content from another page, what's the best thing to do here? In the sake of SEO, remove the duplicate content and only have one? Or won't it make much difference?
Web Design | | vortexuk0 -
Duplicate Product Descriptions for Each Variant
Hi, I am setting up a Shopify e-commerce store and I have a questions about duplicate product descriptions. I have written unique product descriptions for all our products. Each product has at least 10 color options. I am thinking that it would look better if I created each color variant as a unique product. i.e. store.com/nice-shirt-blue, store.com/nice-shirt-red ect. Here is the kicker. Would I be penalized for using the same product descriptions for each product type?
Web Design | | Jon_B0 -
Why is Google Webmaster suddenly started showing hundreds of HTML Improvements
Why is Google Webmaster suddenly started showing hundreds of HTML Improvements I mean to ask, my hundreds pages are been shown as duplicate - despite canonical marked correctly Below are sample url - which are been crawled in own way. I have rechecked canonical tag - which is correct as URL - 1, in all 3 url Do i need to worry about anything or shall i presume its a flaw from search engine to report this as an issue (This only pertain to Forum section) http://www.mycarhelpline.com/index.php?option=com_easydiscuss&view=post&id=1683&Itemid=78 http://www.mycarhelpline.com/?id=1683&Itemid=78&option=com_easydiscuss&view=post http://www.mycarhelpline.com/index.php?option=com_easydiscuss&view=post&id=1683 ps - i know these are dynamic url and not sef friendly url, but its been 3 yrs and , due to our ignorance and site builder took advantage of this. now - nothing can be done much to make them sef friendly as site has several thousand pages and touchwood - these dynamic url are not impacting much
Web Design | | Modi0 -
Does hidden content in jQuery ui tabs still get ignored?
I am looking for a more current answer to this question. I know that google leaves out the js and css. But since the code usually has display:hidden inline with the code while using jquery ui tabs I was curious to know if google considers this hidden or from what some articles have said, "tries to ignore the content". Is this still true today? I would assume no but looking for some back-up.
Web Design | | sknott0 -
Old school HTML and rankings
How does really old school HTML (with inline CSS and a boat load of markup errors) affect modern SEO? I'm talking purely rankings, not conversions or bounce rate etc.
Web Design | | DavidWilsonSEO0 -
Using tables in html
I have a question about tables in html.I heard that you shouldnt use tables in html,you should should use css instead.Ive used free html templates that use tables but those tables are styled through css:td,th,table and other table elements are ale styled through css.I'm curious is this ok for SEO or should tables should be dropped altogether? Thanks for your response
Web Design | | PCTechGuy20120 -
Need advice on diplaying content for Search
Hi every body, I am doing landing page redesign(s). Does any body know or can refer a content carousel that can rotate video and pictures? The "site with images" search option result is a compelling reason to showcase pictures if your space competeiveness (showroom, merchandise, etc) can be improved with a strong image presence. here is my main landing page http://www.shearerpainting.com I know there is alot of stuff, and confusing call to action, but I am looking for strategies to clean it up, clear fous on action (get bid, learn more), but allow users to see that they can dig for more content.
Web Design | | johnshearer0