Duplicate Content - Home Page even wth Mod Rewrite 301
-
Hi,
It looks like Seomoz (and Screaming Frog) is showing my home page as duplicate content.
http://www.mydomain.com Page Authority 61 Linking root Domain 321
http://www.mydomain.com/ Page Authority 61 Linking root Domain 321
[Screaming Frog shows duplicate as]
www.mydomain.com/
www.mydomain.com/index.html}Years ago I hired someone to write the code for a rewrite for non www to be 301 redirected to www version. I was surprised at finding out that I still have a problem.
Here is the code on my htaccess page.
<ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} !^www.mydomain.com [NC]
RewriteRule ^(.*)$ http://www.mydomain.com/$1 [L,R=301]</ifmodule>Was this code not properly written ?
One more question, we were hit hard by Panda and Penguin, would something like this be that much of a factor.
Thanks in advance,
Force7
-
Thanks for the great advice. But once you've added the non-www to www redirect as you wrote above, why not just do this in .htaccess for the ".html to /" issue?
Redirect 301 /index.html http:/www.mydomain.com/
Or
In this case if you've done the ref canonical on he "/" home page, is that good enough or do you still need to redirect /index.html to "/" ?
Thanks!
-
So if I understand correctly, I should have..
<ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} !^www.mydomain.com [NC]
RewriteRule ^(.*)$ http://www.mydomain.com/$1 [L,R=301]</ifmodule>on the .htaccess and then also add
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.index.html\ HTTP/ RewriteRule ^(.)index.html$ /$1 [R=301,L]
AND
RewriteCond %{HTTP_HOST} !^.mydomain.com$ [NC] RewriteRule ^(.+)/$ http://%{HTTP_HOST}/$1 [R=301,L]
The internal linking structure of the site is as follows: main navigation is usually absolute, http://www.domain.com/page.php but throughout the site if I link a keyword i use the "/folder/page.php
When I do a "site" command on Google I see the
www.TranslationSoftware4u.com/ as the only one I saw listed
Our hits are down 70% so I am paranoid about making a mistake during the process of trying to find out how to recover from the latest update.
Appreciate the time you are taking to help answer this Matthew!
Thanks,
Force7
-
Hey, You are solving multiple problems. The code looks properly written to solve one of those problems--the naked domain to www domain redirect. So long as going to http://mydomain.com 301 redirects to www.mydomain.com, then you know that piece is working.
The second "problem" you have is that you can reach your home page with /index.html and without /index.html in the URL. So long as only one is indexed by Google, this isn't that big of a problem. You should however put in a canonical on your home page to make it clear which version you do want indexed. Then make sure all internal links go to that URL.
Alternatively, you can 301 redirect /index.html to the root via the htacess file. That code would go something like this:
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /.index.html\ HTTP/ RewriteRule ^(.)index.html$ /$1 [R=301,L]
The third "problem" is that your home page can be accessed with or without a trailing slash. Again, I wouldn't say this is a problem unless both versions are indexed. And, like the home page, you can fix this by adding a canonical element to the home page and link to that canonical consistently within your site.
However, this too can be fixed via htaccess. Here is an example of the htaccess code:
RewriteCond %{HTTP_HOST} !^.mydomain.com$ [NC] RewriteRule ^(.+)/$ http://%{HTTP_HOST}/$1 [R=301,L]
Thanks,
Matthew
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Content and Subdirectories
Hi there and thank you in advance for your help! I'm seeking guidance on how to structure a resources directory (white papers, webinars, etc.) while avoiding duplicate content penalties. If you go to /resources on our site, there is filter function. If you filter for webinars, the URL becomes /resources/?type=webinar We didn't want that dynamic URL to be the primary URL for webinars, so we created a new page with the URL /resources/webinar that lists all of our webinars and includes a featured webinar up top. However, the same webinar titles now appear on the /resources page and the /resources/webinar page. Will that cause duplicate content issues? P.S. Not sure if it matters, but we also changed the URLs for the individual resource pages to include the resource type. For example, one of our webinar URLs is /resources/webinar/forecasting-your-revenue Thank you!
Technical SEO | | SAIM_Marketing0 -
Purchasing duplicate content
Morning all, I have a client who is planning to expand their product range (online dictionary sites) to new markets and are considering the acquisition of data sets from low ranked competitors to supplement their own original data. They are quite large content sets and would mean a very high percentage of the site (hosted on a new sub domain) would be made up of duplicate content. Just to clarify, the competitor's content would stay online as well. I need to lay out the pros and cons of taking this approach so that they can move forward knowing the full facts. As I see it, this approach would mean forgoing ranking for most of the site and would need a heavy dose of original content as well as supplementing the data on page to build around the data. My main concern would be that launching with this level of duplicate data would end up damaging the authority of the site and subsequently the overall domain. I'd love to hear your thoughts!
Technical SEO | | BackPack851 -
Duplicate content due to numerous sub category level pages
We have a healthcare website which lists doctors based on their medical speciality. We have a paginated series to list hundreds of doctors. Algorithm: A search for Dentist in Newark locality of New York gives a result filled with dentists from Newark followed by list of dentists in locations near by Newark. So all localities under a city have the same set of doctors distributed jumbled an distributed across multiple pages based on nearness to locality. When we don't have any dentists in Newark we populate results for near by localities and create a page. The issue - So when the number of dentists in New York is <11 all Localities X Dentists will have jumbled up results all pointing to the same 10 doctors. The issue is even severe when we see that we have only 1-3 dentists in the city. Every locality page will be exactly the same as a city level page. We have about 2.5 Million pages with the above scenario. **City level page - **https://www.example.com/new-york/dentist - 5 dentists **Locality Level Page - **https://www.example.com/new-york/dentist/clifton, https://www.example.com/new-york/dentist/newark - Page contains the same 5 dentists as in New York city level page in jumbled up or same order. What do you think we must do in such a case? We had discussions on putting a noindex on locality level pages or to apply canonical pointing from locality level to city level. But we are still not 100% sure.
Technical SEO | | ozil0 -
Duplicate Content Due to Pagination
Recently our newly designed website has been suffering from a rankings loss. While I am sure there are a number of factors involved, I'd like to no if this scenario could be harmful... Google is showing a number of duplicate content issues within Webmaster Tools. Some of what I am seeing is duplicate Meta Titles and Meta Descriptions for page 1 and page 2 of some of my product category pages. So if a category has many products and has 4 pages, it is effectively showing the same page title and meta desc. across all 4 pages. I am wondering if I should let my site show, say 150 products per page to get them all on one page instead of the current 36 per page. I use the Big Commerce platform. Thank you for taking the time to read my question!
Technical SEO | | josh3300 -
Tips and duplicate content
Hello, we have a search site that offers tips to help with search/find. These tips are organized on the site in xml format with commas... of course the search parameters are duplicated in the xml so that we have a number of tips for each search parameter. For example if the parameter is "dining room" we might have 35 pieces of advice - all less than a tweet long. My question - will I be penalized for keyword stuffing - how can I avoid this?
Technical SEO | | acraigi0 -
Duplicate Content Issue
Very strange issue I noticed today. In my SEOMoz Campaigns I noticed thousands of Warnings and Errors! I noticed that any page on my website ending in .php can be duplicated by adding anything you want to the end of the url, which seems to be causing these issues. Ex: Normal URL - www.example.com/testing.php Duplicate URL - www.example.com/testing.php/helloworld The duplicate URL displays the page without the images, but all the text and information is present, duplicating the Normal page. I Also found that many of my PDFs seemed to be getting duplicated burried in directories after directories, which I never ever put in place. Ex: www.example.com/catalog/pdfs/testing.pdf/pdfs/another.pdf/pdfs/more.pdfs/pdfs/ ... when the pdfs are only located in a pdfs directory! I am very confused on how to fix this problem. Maybe with some sort of redirect?
Technical SEO | | hfranz0 -
Mod Rewrite question to prevent duplicate content
Hi, I'm having problems with a mod rewrite issue and duplicate content On my website I have Website.com Website.com/directory Website.com/directory/Sub_directory_more_stuff_here Both #1 and #2 are the same page (I can't change this). #3 is different pages. How can I use mod rewrite to to make #2 redirect to #1 so I don't have duplicate content WHILE #3 still works?
Technical SEO | | kat20 -
Duplicate Content Errors
Ok, old fat client developer new at SEO so I apologize if this is obvious. I have 4 errors in one of my campaigns. two are duplicate content and two are duplicate title. Here is the duplicate title error Rare Currency And Old Paper Money Values and Information.
Technical SEO | | Banknotes
http://www.antiquebanknotes.com/ Rare Currency And Old Paper Money Values and Information.
http://www.antiquebanknotes.com/Default.aspx So, my question is... What do I need to do to make this right? They are the same page. in my page load for default.aspx I have this: this.Title = "Rare Currency And Old Paper Money Values and Information."; And it occurs only once...0