Duplicate Content - Home Page even wth Mod Rewrite 301
-
Hi,
It looks like Seomoz (and Screaming Frog) is showing my home page as duplicate content.
http://www.mydomain.com Page Authority 61 Linking root Domain 321
http://www.mydomain.com/ Page Authority 61 Linking root Domain 321
[Screaming Frog shows duplicate as]
www.mydomain.com/
www.mydomain.com/index.html}Years ago I hired someone to write the code for a rewrite for non www to be 301 redirected to www version. I was surprised at finding out that I still have a problem.
Here is the code on my htaccess page.
<ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} !^www.mydomain.com [NC]
RewriteRule ^(.*)$ http://www.mydomain.com/$1 [L,R=301]</ifmodule>Was this code not properly written ?
One more question, we were hit hard by Panda and Penguin, would something like this be that much of a factor.
Thanks in advance,
Force7
-
Thanks for the great advice. But once you've added the non-www to www redirect as you wrote above, why not just do this in .htaccess for the ".html to /" issue?
Redirect 301 /index.html http:/www.mydomain.com/
Or
In this case if you've done the ref canonical on he "/" home page, is that good enough or do you still need to redirect /index.html to "/" ?
Thanks!
-
So if I understand correctly, I should have..
<ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} !^www.mydomain.com [NC]
RewriteRule ^(.*)$ http://www.mydomain.com/$1 [L,R=301]</ifmodule>on the .htaccess and then also add
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.index.html\ HTTP/ RewriteRule ^(.)index.html$ /$1 [R=301,L]
AND
RewriteCond %{HTTP_HOST} !^.mydomain.com$ [NC] RewriteRule ^(.+)/$ http://%{HTTP_HOST}/$1 [R=301,L]
The internal linking structure of the site is as follows: main navigation is usually absolute, http://www.domain.com/page.php but throughout the site if I link a keyword i use the "/folder/page.php
When I do a "site" command on Google I see the
www.TranslationSoftware4u.com/ as the only one I saw listed
Our hits are down 70% so I am paranoid about making a mistake during the process of trying to find out how to recover from the latest update.
Appreciate the time you are taking to help answer this Matthew!
Thanks,
Force7
-
Hey, You are solving multiple problems. The code looks properly written to solve one of those problems--the naked domain to www domain redirect. So long as going to http://mydomain.com 301 redirects to www.mydomain.com, then you know that piece is working.
The second "problem" you have is that you can reach your home page with /index.html and without /index.html in the URL. So long as only one is indexed by Google, this isn't that big of a problem. You should however put in a canonical on your home page to make it clear which version you do want indexed. Then make sure all internal links go to that URL.
Alternatively, you can 301 redirect /index.html to the root via the htacess file. That code would go something like this:
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.index.html\ HTTP/ RewriteRule ^(.)index.html$ /$1 [R=301,L]
The third "problem" is that your home page can be accessed with or without a trailing slash. Again, I wouldn't say this is a problem unless both versions are indexed. And, like the home page, you can fix this by adding a canonical element to the home page and link to that canonical consistently within your site.
However, this too can be fixed via htaccess. Here is an example of the htaccess code:
RewriteCond %{HTTP_HOST} !^.mydomain.com$ [NC] RewriteRule ^(.+)/$ http://%{HTTP_HOST}/$1 [R=301,L]
Thanks,
Matthew
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unavoidable duplicate page
Hi, I have an issue where I need to duplicate content on a new site that I am launching. Visitors to the site need to think that product x is part of two different services. e.g. domain.com/service1/product-x domain.com/service2/product-x Re-writing content for product x for each service section is not an option but possibly I could get over that only one product-x page is indexed by search engines. What's the best way to do this? Any advice would be appreciated. Thanks, Stuart
Technical SEO | | Stuart260 -
Affiliate urls and duplicate content
Hi, What is the best way to get around having an affiliate program, and the affiliate links on your site showing as duplicate content?
Technical SEO | | Memoz0 -
Home page deindexed by google
when I search my website on google by site:www.mydomain.com I have found my domain with www has been de-indexed by google, but when I search site:mydomain.com, my home page--**mydomain.com **show up on the search results without www, put it simple, google only index my domain without www, I wonder how to make my domain with www being indexed, and how to prevent this problem occure again.
Technical SEO | | semer0 -
Is content important on home page
hi. i am working on a site at the moment www.in2town.co.uk and i am trying to decide if on the second column of my site where it says uk news, if i should keep it the way it is and have content under the picture or should i get rid of the content under the picture and just have the main title. I am wanting to know if the content under the picture is important for google and for the reader or would it be better just to have the title which is h2. any help would be great.
Technical SEO | | ClaireH-1848860 -
Duplicate Content
The crawl shows a lot of duplicate content on my site. Most of the urls its showing are categories and tags (wordpress). so what does this mean exactly? categories is too much like other categories? And how do i go about fixing this the best way. thanks
Technical SEO | | vansy0 -
What can be the cause of my inner pages ranking higher than my home page?
If you do a search for my own company name or products we sell the inner pages rank higher than the homepage and if you do a search for exact content from my home page my home page doesn't show in the results. My homepage shows when you do a site: search so not sure what is causing this.
Technical SEO | | deciph220 -
Are all duplicate pages bad?
I just got my first Crawl Report for my forum and it said I have almost 9,000 duplicate pages. When I looked at a sample of them though I saw that many of them were "reply" links. By this I mean the "reply" button was clicked for a topic yet since the crawler was not a member, it just brought them to the login/register screen. Since all the topics would bring you to the same login page I'm assuming it counted all these "reply" links as duplicates. Should I just ignore these or is there some way to fix it? Thanks in advance.
Technical SEO | | Xee0 -
Crawl Errors and Duplicate Content
SEOmoz's crawl tool is telling me that I have duplicate content at "www.mydomain.com/pricing" and at "www.mydomain.com/pricing.aspx". Do you think this is just a glitch in the crawl tool (because obviously these two URL's are the same page rather than two separate ones) or do you think this is actually an error I need to worry about? Is so, how do I fix it?
Technical SEO | | MyNet0