Duplicate Content - Home Page even wth Mod Rewrite 301
-
Hi,
It looks like Seomoz (and Screaming Frog) is showing my home page as duplicate content.
http://www.mydomain.com Page Authority 61 Linking root Domain 321
http://www.mydomain.com/ Page Authority 61 Linking root Domain 321
[Screaming Frog shows duplicate as]
www.mydomain.com/
www.mydomain.com/index.html}Years ago I hired someone to write the code for a rewrite for non www to be 301 redirected to www version. I was surprised at finding out that I still have a problem.
Here is the code on my htaccess page.
<ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} !^www.mydomain.com [NC]
RewriteRule ^(.*)$ http://www.mydomain.com/$1 [L,R=301]</ifmodule>Was this code not properly written ?
One more question, we were hit hard by Panda and Penguin, would something like this be that much of a factor.
Thanks in advance,
Force7
-
Thanks for the great advice. But once you've added the non-www to www redirect as you wrote above, why not just do this in .htaccess for the ".html to /" issue?
Redirect 301 /index.html http:/www.mydomain.com/
Or
In this case if you've done the ref canonical on he "/" home page, is that good enough or do you still need to redirect /index.html to "/" ?
Thanks!
-
So if I understand correctly, I should have..
<ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} !^www.mydomain.com [NC]
RewriteRule ^(.*)$ http://www.mydomain.com/$1 [L,R=301]</ifmodule>on the .htaccess and then also add
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.index.html\ HTTP/ RewriteRule ^(.)index.html$ /$1 [R=301,L]
AND
RewriteCond %{HTTP_HOST} !^.mydomain.com$ [NC] RewriteRule ^(.+)/$ http://%{HTTP_HOST}/$1 [R=301,L]
The internal linking structure of the site is as follows: main navigation is usually absolute, http://www.domain.com/page.php but throughout the site if I link a keyword i use the "/folder/page.php
When I do a "site" command on Google I see the
www.TranslationSoftware4u.com/ as the only one I saw listed
Our hits are down 70% so I am paranoid about making a mistake during the process of trying to find out how to recover from the latest update.
Appreciate the time you are taking to help answer this Matthew!
Thanks,
Force7
-
Hey, You are solving multiple problems. The code looks properly written to solve one of those problems--the naked domain to www domain redirect. So long as going to http://mydomain.com 301 redirects to www.mydomain.com, then you know that piece is working.
The second "problem" you have is that you can reach your home page with /index.html and without /index.html in the URL. So long as only one is indexed by Google, this isn't that big of a problem. You should however put in a canonical on your home page to make it clear which version you do want indexed. Then make sure all internal links go to that URL.
Alternatively, you can 301 redirect /index.html to the root via the htacess file. That code would go something like this:
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.index.html\ HTTP/ RewriteRule ^(.)index.html$ /$1 [R=301,L]
The third "problem" is that your home page can be accessed with or without a trailing slash. Again, I wouldn't say this is a problem unless both versions are indexed. And, like the home page, you can fix this by adding a canonical element to the home page and link to that canonical consistently within your site.
However, this too can be fixed via htaccess. Here is an example of the htaccess code:
RewriteCond %{HTTP_HOST} !^.mydomain.com$ [NC] RewriteRule ^(.+)/$ http://%{HTTP_HOST}/$1 [R=301,L]
Thanks,
Matthew
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Pages with Duplicate Content
When I crawl my site through moz, it shows lots of Pages with Duplicate Content. The thing is all that pages are pagination pages. How should I solve this issue?
Technical SEO | | 100offdeal0 -
To 301 or not to 301?
So, to cut a long story short, on our website we have a /product page that is very similar to our homepage and doesn't really serve much of a purpose. It doesn't really fit in with the rest of the website and our directors want to get rid of it and focus our efforts on our homepage. Problem is, the /product page has a little bit of PA and links to other important pages on the website. I personally don't want to completely cut this page off. Would a 301 redirect to the homepage be a good option or would I be better off redirecting users to our course library page (Our course pages are what bring in most of our organic traffic)? Any help or other suggestions would be appreciated here! http://www.ihasco.co.uk/
Technical SEO | | iHasco1 -
Is this duplicate content when there is a link back to the original content?
Hello, My question is: Is it duplicate content when there is a link back to the original content? For example, here is the original page: http://www.saugstrup.org/en-ny-content-marketing-case-infografik/. But that same content can be found here: http://www.kommunikationsforum.dk/anders-saugstrup/blog/en-ny-content-marketing-case-til-dig, but there is a link back to the original content. Is it still duplicate content? Thanks in advance.
Technical SEO | | JoLindahl912 -
Duplicated content on subcategory pages: how do I fix it?
Hello Everybody,
Technical SEO | | uMoR
I manage an e-commerce website and we have a duplicated content issue for subcategory. The scenario is like this: /category1/subcategory1
/category2/subcategory1
/category3/subcategory1 A single subcategory can fit multiple categories, so we have 3 different URL for the same subcategory with the same content (except of the navigation link). Which are the best practice to avoid this issue? Thank you!0 -
Category URL Duplicate Content
I've recently been hired as the web developer for a company with an existing web site. Their web architecture includes category names in product urls, and of course we have many products in multiple categories thus generating duplicate content. According to the SEOMoz Site Crawl, we have roughly 1600 pages of duplicate content, I expect primarily from this issue. This is out of roughly 3600 pages crawled. My questions are: 1. Fixing this for the long term will obviously mean restructuring the URLs for the site. Is this worthwhile and what will the ramifications be of performing such a move? 2. How can I determine the level and extent of the effects of this duplicated content? 3. Is it possible the best course of action is to do nothing? The site has many, many other issues, and I'm not sure how highly to prioritize this problem. In addition, the IT man is highly doubtful this is causing an SEO issue, and I'm going to need to be able to back up any action I request. I do feel I will need to strongly justify any possible risks this level of site change could cause. Thanks in advance, and please let me know if any more information is needed.
Technical SEO | | MagnetsUSA0 -
Pages with different content and meta description marked as duplicate content
I am running into an issue where I have pages with completely different body and meta description but they are still being marked as having the same content (Duplicate Page Content error). What am I missing here? Examples: http://www.wallstreetoasis.com/forums/what-to-expect-in-the-summer-internship
Technical SEO | | WallStreetOasis.com
and
http://www.wallstreetoasis.com/blog/something-ventured http://www.wallstreetoasis.com/forums/im-in-the-long-run
and
http://www.wallstreetoasis.com/image/jhjpeg0 -
What's the best way to eliminate duplicate page content caused by blog archives?
I (obviously) can't delete the archived pages regardless of how much traffic they do/don't receive. Would you recommend a meta robot or robot.txt file? I'm not sure I'll have access to the root directory so I could be stuck with utilizing a meta robot, correct? Any other suggestions to alleviate this pesky duplicate page content issue?
Technical SEO | | ICM0 -
Duplicate Page Issue
Dear All, I am facing stupid duplicate page issue, My whole site is in dynamic script and all the URLs were in dynamic, So i 've asked my programmer make the URLs user friendly using URL Rewrite, but he converted aspx pages to htm. And the whole mess begun. Now we have 3 different URLs for single page. Such as: http://www.site.com/CityTour.aspx?nodeid=4&type=4&id=47&order=0&pagesize=4&pagenum=4&val=Multi-Day+City+Tours http://www.tsite.com/CityTour.aspx?nodeid=4&type=4&id=47&order=0&pagesize=4&pagenum=4&val=multi-day-city-tours http://www.site.com/city-tour/multi-day-city-tours/page4-0.htm I think my programmer messed up the URL Rewrite in ASP.net(Nginx) or even didn't use it. So how do i overcome this problem? Should i add canonical tag in both dynamic URLs with pointing to pag4-0.htm. Will it help? Thanks!
Technical SEO | | DigitalJungle0