Duplicate Content for index.html
-
In the Crawl Diagnostics Summary, it says that I have two pages with duplicate content which are:
I read in a Dream Weaver tutorial that you should name your home page "index.html" and then you can let www.mywebsite.com automatically direct the user to index.html. Is this a bug in SEOMoz's crawler or is it a real problem with my site?
Thank you,
Dan
-
The code should definitely go into the websites root directory's .htaccess, however .htaccess can be weird, a few days ago I ran into a similar issue with a client's website, and I was able to remedy the issue with a variation of the code.
index Redirect RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)index.(php|html|htm|asp)\ HTTP/ RewriteRule ^(([^/]+/))index.(php|html|htm|asp)$ http://yoursite.com/$1 [R=301,L]
If you give me the URL for the site I will take a look at it and let you know what would be feasible.
-
Hi Daniel, can you share with us the URL of your site? We can take a look at it and give you a more precise answer that way. Thanks!
-
I eventually figured out that your method was a 301 redirect and I definitely broke my site trying to use the code you posted. .. haha. Its ok though. I just removed the code and it went back to normal. At first, I was editing the .htaccess file in the public_html folder which wasnt working. Then I tried the root folder for the site (I created the .htaccess file since it did not exist.) Neither of those worked. (I am using Bluehost so I do not think that I have root access and I am not sure if it is a Linux server or not.)
If there is an easy way to explain what I am doing wrong, please do so. Otherwise, I will use canonical.
Thanks for everything!
-
@Dan
Thanks for your reply. It seems like there are lots of different ways to solve this problem. I just watched this video on Matt Cutt's blog where he discusses his preference for 301 redirects over rel canonical tag.
Where would you say your solution fits in?
sorry about the delay of this response, i didn't realize the that you were asking me a question right away. When placing the code I provided in my previous answer this will cause a 301 perminant redirect to the original URL. That's actually what the
[R=301,L]
portion of the code is stating (R) redirect (301) status is referring to. After reviewing the Matt Cutts video, I realize that I should have asked you if you were operating on a Linux server that you had root access to. We actually utilize both redirects and canonical tags since it was recommended by the on-page optimization reports. Heck Google uses them, I would assume because it's easier for the user to be referred to a single page URL. Obviously though if you don't have server header access, and are not familiar with .htaccess (you can accidentally break your site) then the canonical solution is appropriate
-
Josh,
Thanks for your reply. It seems like there are lots of different ways to solve this problem. I just watched this video on Matt Cutt's blog where he discusses his preference for 301 redirects over rel canonical tag.
Where would you say your solution fits in?
Thanks,
Dan -
use the link rel tag for all my homepages for the http://www.yoursite.com
-
Odd enough I just recently answered this question. The SEOmoz crawler is correct, because without a redirect you will be able to access both versions of the page in your browser.
To resolve this issue simply rewrite the index.html to the root url by placing the following code into your .htaccess file into your root directory.
Options +FollowSymlinks RewriteEngine on
Index Rewrite RewriteRule ^index.(htm|html|php) http://www.yoursite.com/ [R=301,L] RewriteRule ^(.*)/index.(htm|html|php) http://www.yoursite.com/$1/ [R=301,L]
You can also do the same with the index file in any subdirectories that you might create, by simply placing a .htaccess into those sub directories and using variations of the above code. This is how you create nice tight URLs without the duplicate content issue that look like - http://www.semclix.com/design/business/
-
It is a problem which you need to fix. You need to canonicalize your pages.
Those are all various URLs which most likely lead to the same web page. I say "most likely" because these URLs can actually lead to different pages.
You need to tell crawlers and search engines how you organize your site. There are several ways to achieve canonicalization. The method I prefer is to add the following line of code to each page:
The URL provided should be the preferred URL for your page.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why did Google Index a weird version of my blog post?
i wrote a page - https://domain.com/how-to-do-xyz/ but when doing an inurl search, i see that it is indexed by google as - https://secureservercdn.net/58584.883848.9834983/myftpupload/how-to-do-xyz/ (not actual url) and when i view that page, it is a weirdly formatted version of the page with many design elements missing. this is a wordpress site. Why would this be? thanks, Ryan
Web Design | | RyanMeighan0 -
Requirements for mobile menu design have created a duplicated menu in the text/cache view.
Hi, Upon checking the text cache view of our home page, I noticed the main menu has been duplicated. Please see: http://webcache.googleusercontent.com/search?q=cache:http://www.trinitypower.com&strip=1 Our coder tells me he created one version for the desktop and one for the mobile version. Duplicating the menu cannot be good for on page SEO. With that said, I have had no warnings reported back from Moz. Maybe the moz bots are not tuned to looks for such a duplication error. Anyway, the reason the coder created a different menu for mobile in order to support the design requirements. I did not like the look and feel of the responsive version created based on the desktop version. Hi solution to this problem is to convert the Mobile version menu into ajax. what do you guys think? Thanks, Jarrett
Web Design | | TrinityPower0 -
Duplicate items across different pages?
On our new website we have a testimonials page which you can cycle through them. We also have the testimonial on the our work / project page. Essentially this is duplicate content from another page, what's the best thing to do here? In the sake of SEO, remove the duplicate content and only have one? Or won't it make much difference?
Web Design | | vortexuk0 -
Traffic flow going through homepage, /default.aspx, and /index.htm - How do I fix this?
In the Google Analytics visitor flow section, most of my traffic is starting off on the homepage, but a significant portion somehow is being directed to /default.aspx, as well as a minor fraction at /index.htm - see the attached image for what this looks like. My questions are, why is this happening, is it negatively affecting the site, and should I just fix it through a filter in Analytics or consult IT? Navigating to the /default.aspx page in the address bar simply returns a blank screen. Any idea of what's going on here? The logo in the top left corner of the site directs to /index.htm, so I think I have that issue solved, but /default.aspx is definitely stumping me. The server currently runs on Apache, though I don't know if it always did. My standard method of simply Googling the problem didn't give me any definitive answers, so any help would be greatly appreciated. Q12QqpC.jpg
Web Design | | BD690 -
Redirect From .aspx to .html if already indexed - Website Redesign
Hi Guys I would like to know if somebody could possibly shed some light on this for me. We are in the process of re-designing our site, but we are keeping all of our content in terms of site structure, internal linking etc. the same. Now we were wondering if it would be a SEO best practice for us to change our pages' extension from .aspx to .html and just put a re-direct from the aspx to the html pages. Or should we keep everything as is, and maybe just revise our on-page seo efforts as well as do some more link-building. I just have to note that we are currently ranking very well for top positions and obviously all these pages are already nicely indexed. And then another question I have is with regards to our mobi site of this same website.Our dev team created it using Responsive Web Design, but they decided to implement techniques that show and hide content based on what device you are viewing it on. So when viewing it on your desktop, it will show content as per normal, but when viewing it on a mobile device it will hide this content and show the content formatted for that specific mobile device. So we are obviously sitting with a case of dup content here.Is this technique acceptable, or is there a workaround/different way of implementing this? Thanks In Advance Dave
Web Design | | DavidZA10 -
Duplicate Page Title
Virtually all of my pages are coming up with a "Duplicate Page Title" error even though the page title are different. I assume this is down to the end of the page title having the company name. Is this the reason and is it a problem to have a page title like below... "Page title description - Company Name"
Web Design | | petewinter0 -
Suggestions for content slider/image slider copy/paste application.
Hey Moz Community, I am looking for a content slider that can be easily changed by non-technicals for posting different styles of content/calls to action and this seems to be best: http://www.slidedeck.com/ I have installed a nivo slider on a Seattle Painting site, and flash slider on a commercial painting site. But I want my blog clients to be able to format..then copy/paste code..linke embedding a video. opinions? Thanks John
Web Design | | johnshearer0 -
Using "#" anchors to display different content
If I have a page that has an area on the page that acts like a widget and has three different tabs. These tabs provide 3 different types of information relevant to the page subject matter. By default when someone goes to the page one of the tabs is showing but you have to click on the others to see the info on them. Is it OK to use domain.com/topic#TAB1, domain.com/topic#TAB2, domain.com/topic#TAB3 to create shortcut links so that people can land on the page and have that predetermined tab showing. I'm wondering what search engines might think. Essentially all the content of all three tabs is there for people to see but they'd have to click to see the other tabs. I don't consider the content to be hidden. But I'd like to hear people's thoughts.
Web Design | | Business.com0