Duplicate Content Issue
-
Why do URL with .html or index.php at the end are annoying to the search engine? I heard it can create some duplicate content but I have no idea why? Could someone explain me why is that so?
Thank you
-
Using this as an example: If google showed duplicate content for http://www.domain.com/seo.html and http://www.domain.com/seo then you would want a 301 .htaccess redirect so anyone accessing the http://www.domain.com/seo.htm version of the page is automatically sent to http://www.domain.com/seo.
You can use the following in your .htaccess file.
RewriteRule ^/seo.html$ http://www.domain.com/seo/? [R=301,NC,L]
The benefit to having no file extension (.html) on the end of your URLs is it allows you to change the underlying framework or your website or even the programming language without the need for adding redirects each time you change (provided the website structure remains the same)
-
If there is the same page with the urls domain.com/seo & domain.com/seo.htm for this is what can be considered as duplicate content. With or without (better without) there should only be one version of the URL, to not split the link juice passing through. Hope this helps, Vahe
-
Agreed
-
There is also the problem with http://www.example.com and http:/example.com - they also generate duplicate content.
What you should do?
First - solve the http://www.example vs http://example issue
Edit your .htaccess file (if you don't have it, create one). Inside this file you need to input:RewriteEngine On RewriteCond %{HTTP_HOST} ^example.com RewriteRule (.*) http://www.example.com/$1 [R=301,L]
(where example.com is your websiteSecond - solve the index.html duplicate content
You need to include in the index.html file the follow metag inside the section:
This tag will tell google to forget the index.html and focus at www.example.com. So you will avoid the duplicate content without any problem
I hope I could help you.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Upper and lower case URLS coming up as duplicate content
Hey guys and gals, I'm having a frustrating time with an issue. Our site has around 10 pages that are coming up as duplicate content/ duplicate title. I'm not sure what I can do to fix this. I was going to attempt to 301 direct the upper case to lower but I'm worried how this will affect our SEO. can anyone offer some insight on what I should be doing? Update: What I'm trying to figure out is what I should do for our URL's. For example, when I run an audit I'm getting two different pages: aaa.com/BusinessAgreement.com and also aaa.com/businessagreement.com. We don't have two pages but for some reason, Google thinks we do.
Intermediate & Advanced SEO | | davidmac1 -
Multiple H2 with no direct links to content and invisible body text - is this an issue?
OK, so we've lost pagerank and I think it's because of the way our site works (and we operate it). We have a videofolio, which shows on most of our pages, showcasing our work. Over time, we have tended to unpublish these pages and created new videofolio pages to show on our home page and relevant pages. The videofolio is a set of pages, each with a title, body text and a place to insert a link to the video, which is played through a videofolio showcase on our website (www.curveball-media.co.uk). Each is set a category, e.g. film, and when the user clicks the tab for 'film', the thumbnails pop up and the user can play the video. We have to work it this way as it's the only way to remove the videos from showing on our home page and to show new content instead. Simply deselecting a category still allows the videos to be seen when the 'all' category is selected by the user. Last week, I found a way of bringing back these unpublished pages by removing the 'all' tab from the videofolio. Then I turned each one into a blog like structure instead. Essentially, instead of the video link being played through the videofolio, we deselected a category (e.g. animation, film etc) and left the page floating. The only way you can access it without being attached to a videofolio category is through the direct link. By turning off the 'all' category and deselecting the page from any other categories, we were able to properly SEO these pages. NB: If they are created for use with the videofolio, you can have only extremely limited body text and no H2, as this is the text that appears when you hover over the video thumbnail. That's just the nature of the template. What I didn't anticipate is that now the code on the home page shows all these now (un)published pages and their corresponding H2 tags. Without a category selected, there is no way to get to these pages unless I create a direct link. I plan to do this through a blog post. In the home page code, the entire videofolio page shows, including the body text and link to the video. **This text doesn't show on the home page though, i.e. the user never sees this text. ** 1. Is it an issue to have so many similar H2 tags on the homepage? 2. Is it an issue that the code has text which is essentially invisible on the home page? 3. Is it an issue that the content is not linked to through the home page visibly? Thanks!
Intermediate & Advanced SEO | | curveballmedia0 -
How bad is duplicate content for ecommerce sites?
We have multiple eCommerce sites which not only share products across domains but also across categories within a single domain. Examples: http://www.artisancraftedhome.com/sinks-tubs/kitchen-sinks/two-tone-sinks/medium-rounded-front-farmhouse-sink-two-tone-scroll http://www.coppersinksonline.com/copper-kitchen-and-farmhouse-sinks/two-tone-kitchen-farmhouse-sinks/medium-rounded-front-farmhouse-sink-two-tone-scroll http://www.coppersinksonline.com/copper-sinks-on-sale/medium-rounded-front-farmhouse-sink-two-tone-scroll We have selected canonical links for each domain but I need to know if this practice is having a negative impact on my SEO.
Intermediate & Advanced SEO | | ArtisanCrafted0 -
Semi-duplicate content yet authoritative site
So I have 5 real estate sites. One of those sites is of course the original, and it has more/better content on most of the pages than the other sites. I used to be top ranked for all of the subdivsion names in my town. Then when I did the next 2-4 sites, I had some sites doing better than others for certain keywords, and then I have 3 of those sites that are basically the same URL structures (besides the actual domain) and they aren't getting fed very many visits. I have a couple of agents that work with me that I loaned my sites to to see if that would help since it would be a different name. My same youtube video is on each of the respective subdivision pages of my site and theirs. Also, their content is just rewritten content from mine about the same length of content. I have looked over and seen a few of my competitors who only have one site and their URL structures arent good at all, and their content isn't good at all and a good bit of their pages rank higher than my main site which is very frustrating to say the least since they are actually copy cats to my site. I sort of started the precedent of content, mapping the neighborhood, how far that subdivision is from certain landmarks, and then shot a video of each. They have pretty much done the same thing and are now ahead of me. What sort of advice could you give me? Right now, I have two sites that are almost duplicate in terms of a template and same subdivsions although I did change the content the best I could, and that site is still getting pretty good visits. I originally did it to try and dominate the first page of the SERPS and then Penguin and Panda came out and seemed to figure that game out. So now, I would still like to keep all the sites, but I'm assuming that would entail making them all unique, which seems to be tough seeing as though my town has the same subdivisions. Curious as to what the suggestions would be, as I have put a lot of time into these sites. If I post my site will it show up in the SERPS? Thanks in advance
Intermediate & Advanced SEO | | Veebs0 -
Can a website be punished by panda if content scrapers have duplicated content?
I've noticed recently that a number of content scrapers are linking to one of our websites and have the duplicate content on their web pages. Can content scrapers affect the original website's ranking? I'm concerned that having duplicated content, even if hosted by scrapers, could be a bad signal to Google. What are the best ways to prevent this happening? I'd really appreciate any help as I can't find the answer online!
Intermediate & Advanced SEO | | RG_SEO0 -
Duplicate Content For E-commerce
On our E-commerce site, we have multiple stores. Products are shown on our multiple stores which has created a duplicate content problem. Basically if we list a product say a shoe,that listing will show up on our multiple stores I assumed the solution would be to redirect the pages, use non follow tags or to use the rel=canonical tag. Are there any other options for me to use. I think my best bet is to use a mixture of 301 redirects and canonical tags. What do you recommend. I have 5000+ pages of duplicate content so the problem is big. Thanks in advance for your help!
Intermediate & Advanced SEO | | pinksgreens0 -
Duplicate peices of content on multiple pages - is this a problem
I have a couple of WordPress clients with the same issue but caused in different ways: 1. The Slash WP theme which is a portfolio theme, involves setting up multiple excerpts of content that can then be added to multiple pages. So although the pages themselves are not identical, there are the same snippets of content appearing on multiple pages 2. A WP blog which has multiple categories and/or tags for each post, effectively ends up with many pages showing duplicate excerpts of content. My view has always been to noindex these pages (via Yoast), but was advised recently not to. In both these cases, even though the pages are not identical, do you think this duplicate content across multiple pages could cause an issue? All thoughts appreciated
Intermediate & Advanced SEO | | Chammy0 -
Is this duplicate content something to be concerned about?
On the 20th February a site I work on took a nose-dive for the main terms I target. Unfortunately I can't provide the url for this site. All links have been developed organically so I have ruled this out as something which could've had an impact. During the past 4 months I've cleaned up all WMT errors and applied appropriate redirects wherever applicable. During this process I noticed that mydomainname.net contained identical content to the main mydomainname.com site. Upon discovering this problem I 301 redirected all .net content to the main .com site. Nothing has changed in terms of rankings since doing this about 3 months ago. I also found paragraphs of duplicate content on other sites (competitors in different countries). Although entire pages haven't been copied there is still enough content to highlight similarities. As this content was written from scratch and Google would've seen this within it's crawl and index process I wanted to get peoples thoughts as to whether this is something I should be concerned about? Many thanks in advance.
Intermediate & Advanced SEO | | bfrl0