Skip to content
    Moz logo Menu open Menu close
    • Products
      • Moz Pro
      • Moz Pro Home
      • Moz Local
      • Moz Local Home
      • STAT
      • Moz API
      • Moz API Home
      • Compare SEO Products
      • Moz Data
    • Free SEO Tools
      • Domain Analysis
      • Keyword Explorer
      • Link Explorer
      • Competitive Research
      • MozBar
      • More Free SEO Tools
    • Learn SEO
      • Beginner's Guide to SEO
      • SEO Learning Center
      • Moz Academy
      • MozCon
      • Webinars, Whitepapers, & Guides
    • Blog
    • Why Moz
      • Digital Marketers
      • Agency Solutions
      • Enterprise Solutions
      • Small Business Solutions
      • The Moz Story
      • New Releases
    • Log in
    • Log out
    • Products
      • Moz Pro

        Your all-in-one suite of SEO essentials.

      • Moz Local

        Raise your local SEO visibility with complete local SEO management.

      • STAT

        SERP tracking and analytics for enterprise SEO experts.

      • Moz API

        Power your SEO with our index of over 44 trillion links.

      • Compare SEO Products

        See which Moz SEO solution best meets your business needs.

      • Moz Data

        Power your SEO strategy & AI models with custom data solutions.

      Let your reputation grow with Reviews AI
      Moz Local

      Let your reputation grow with Reviews AI

      Learn more
    • Free SEO Tools
      • Domain Analysis

        Get top competitive SEO metrics like DA, top pages and more.

      • Keyword Explorer

        Find traffic-driving keywords with our 1.25 billion+ keyword index.

      • Link Explorer

        Explore over 40 trillion links for powerful backlink data.

      • Competitive Research

        Uncover valuable insights on your organic search competitors.

      • MozBar

        See top SEO metrics for free as you browse the web.

      • More Free SEO Tools

        Explore all the free SEO tools Moz has to offer.

      NEW Keyword Suggestions by Topic
      Moz Pro

      NEW Keyword Suggestions by Topic

      Learn more
    • Learn SEO
      • Beginner's Guide to SEO

        The #1 most popular introduction to SEO, trusted by millions.

      • SEO Learning Center

        Broaden your knowledge with SEO resources for all skill levels.

      • On-Demand Webinars

        Learn modern SEO best practices from industry experts.

      • How-To Guides

        Step-by-step guides to search success from the authority on SEO.

      • Moz Academy

        Upskill and get certified with on-demand courses & certifications.

      • MozCon

        Save on Early Bird tickets and join us in London or New York City

      Unlock flexible pricing & new endpoints
      Moz API

      Unlock flexible pricing & new endpoints

      Find your plan
    • Blog
    • Why Moz
      • Digital Marketers

        Simplify SEO tasks to save time and grow your traffic.

      • Small Business Solutions

        Uncover insights to make smarter marketing decisions in less time.

      • Agency Solutions

        Earn & keep valuable clients with unparalleled data & insights.

      • Enterprise Solutions

        Gain a competitive edge in the ever-changing world of search.

      • The Moz Story

        Moz was the first & remains the most trusted SEO company.

      • New Releases

        Get the scoop on the latest and greatest from Moz.

      Surface actionable competitive intel
      New Feature

      Surface actionable competitive intel

      Learn More
    • Log in
      • Moz Pro
      • Moz Local
      • Moz Local Dashboard
      • Moz API
      • Moz API Dashboard
      • Moz Academy
    • Avatar
      • Moz Home
      • Notifications
      • Account & Billing
      • Manage Users
      • Community Profile
      • My Q&A
      • My Videos
      • Log Out

    The Moz Q&A Forum

    • Forum
    • Questions
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. Home
    2. Digital Marketing
    3. Web Design
    4. Duplicate Content for index.html

    Moz Q&A is closed.

    After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

    Duplicate Content for index.html

    Web Design
    5
    9
    4866
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with question management privileges can see it.
    • superTallDan
      superTallDan last edited by

      In the Crawl Diagnostics Summary, it says that I have two pages with duplicate content which are:

      www.mywebsite.com/

      www.mywebsite.com/index.html

      I read in a Dream Weaver tutorial that you should name your home page "index.html" and then you can let www.mywebsite.com automatically direct the user to index.html.  Is this a bug in SEOMoz's crawler or is it a real problem with my site?

      Thank you,

      Dan

      1 Reply Last reply Reply Quote 0
      • SEMCLIX
        SEMCLIX @superTallDan last edited by

        The code should definitely go into the websites root directory's .htaccess, however .htaccess can be weird, a few days ago I ran into a similar issue with a client's website, and I was able to remedy the issue with a variation of the code.

        index Redirect RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)index.(php|html|htm|asp)\ HTTP/ RewriteRule ^(([^/]+/))index.(php|html|htm|asp)$ http://yoursite.com/$1 [R=301,L]

        If you give me the URL for the site I will take a look at it and let you know what would be feasible.

        1 Reply Last reply Reply Quote 0
        • KeriMorgret
          KeriMorgret @superTallDan last edited by

          Hi Daniel, can you share with us the URL of your site? We can take a look at it and give you a more precise answer that way. Thanks!

          1 Reply Last reply Reply Quote 0
          • superTallDan
            superTallDan @SEMCLIX last edited by

            I eventually figured out that your method was a 301 redirect and I definitely broke my site trying to use the code you posted. .. haha.  Its ok though.  I just removed the code and it went back to normal.  At first, I was editing the .htaccess file in the public_html folder which wasnt working.  Then I tried the root folder for the site (I created the .htaccess file since it did not exist.)  Neither of those worked.  (I am using Bluehost so I do not think that I have root access and I am not sure if it is a Linux server or not.)

            If there is an easy way to explain what I am doing wrong, please do so.  Otherwise, I will use canonical.

            Thanks for everything!

            KeriMorgret SEMCLIX 2 Replies Last reply Reply Quote 0
            • SEMCLIX
              SEMCLIX last edited by

              @Dan

              Thanks for your reply.  It seems like there are lots of different ways to solve this problem.  I just watched this video on Matt Cutt's blog where he discusses his preference for 301 redirects over rel canonical tag.

              Where would you say your solution fits in?

              sorry about the delay of this response, i didn't realize the that you were asking me a question right away. When placing the code I provided in my previous answer this will cause a 301 perminant redirect to the original URL. That's actually what the

              [R=301,L]

              portion of the code is stating (R) redirect (301) status is referring to. After reviewing the Matt Cutts video, I realize that I should have asked you if you were operating on a Linux server  that you had root access to. We actually utilize both redirects and canonical tags since it was recommended by the on-page optimization reports. Heck Google uses them, I would assume because it's easier for the user to be referred to a single page URL. Obviously though if you don't have server header access, and are not familiar with .htaccess (you can accidentally break your site) then the canonical solution is appropriate

              superTallDan 1 Reply Last reply Reply Quote 0
              • superTallDan
                superTallDan @SEMCLIX last edited by

                Josh,

                Thanks for your reply.  It seems like there are lots of different ways to solve this problem.  I just watched this video on Matt Cutt's blog where he discusses his preference for 301 redirects over rel canonical tag.

                Where would you say your solution fits in?

                Thanks,
                Dan

                watch?v=zW5UL3lzBOA

                1 Reply Last reply Reply Quote 0
                • usaccess608
                  usaccess608 last edited by

                  use the link rel tag for all my homepages for the http://www.yoursite.com

                  1 Reply Last reply Reply Quote 0
                  • SEMCLIX
                    SEMCLIX last edited by

                    Odd enough I just recently answered this question. The SEOmoz crawler is correct, because without a redirect you will be able to access both versions of the page in your browser.

                    www.yoursite.com/

                    www.yoursite.com/index.html

                    To resolve this issue simply rewrite the index.html to the root url by placing the following code into your .htaccess file into your root directory.

                    Options +FollowSymlinks                                                                                                            RewriteEngine on

                    Index Rewrite                                                                                                                                RewriteRule ^index.(htm|html|php) http://www.yoursite.com/ [R=301,L]                                      RewriteRule ^(.*)/index.(htm|html|php) http://www.yoursite.com/$1/ [R=301,L]

                    You can also do the same with the index file in any subdirectories that you might create, by simply placing a .htaccess into those sub directories and using variations of the above code. This is how you create nice tight URLs without the duplicate content issue that look like - http://www.semclix.com/design/business/

                    superTallDan 1 Reply Last reply Reply Quote 2
                    • RyanKent
                      RyanKent last edited by

                      It is a problem which you need to fix. You need to canonicalize your pages.

                      www.mywebsite.com

                      www.mywebsite.com/

                      www.mywebsite.com/index.html

                      mywebsite.com/

                      mywebsite.com/index.html

                      Those are all various URLs which most likely lead to the same web page. I say "most likely" because these URLs can actually lead to different pages.

                      You need to tell crawlers and search engines how you organize your site. There are several ways to achieve canonicalization. The method I prefer is to add the following line of code to each page:

                      The URL provided should be the preferred URL for your page.

                      1 Reply Last reply Reply Quote 0
                      • 1 / 1
                      • First post
                        Last post

                      Got a burning SEO question?

                      Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


                      Start my free trial


                      Browse Questions

                      Explore more categories

                      • Moz Tools

                        Chat with the community about the Moz tools.

                      • SEO Tactics

                        Discuss the SEO process with fellow marketers

                      • Community

                        Discuss industry events, jobs, and news!

                      • Digital Marketing

                        Chat about tactics outside of SEO

                      • Research & Trends

                        Dive into research and trends in the search industry.

                      • Support

                        Connect on product support and feature requests.

                      • See all categories

                      Related Questions

                      • Sam.at.Moz

                        Hiding content until user scrolls - Will Google penalize me?

                        I've used: "opacity:0;" to hide sections of my content, which are triggered to show (using Javascript) once the user scrolls over these sections. I remember reading a while back that Google essentially ignores content which is hidden from your page (it mentioned they don't index it, so it's close to impossible to rank for it). Is this still the case? Thanks, Sam

                        Web Design | | Sam.at.Moz
                        0
                      • Dino64

                        Duplicate Content Issue: Mobile vs. Desktop View

                        Setting aside my personal issue with Google's favoritism for Responsive websites, which I believe doesn't always provide the best user experience, I have a question regarding duplicate content... I created a section of a Wordpress web page (using Visual Composer) that shows differently on mobile than it does on desktop view. This section has the same content for both views, but is formatted differently to give a better user experience on mobile devices. I did this by creating two different text elements, formatted differently, but containing the same content. The problem is that both sections appear in the source code of the page. According to Google, does that mean I have duplicate content on this page?

                        Web Design | | Dino64
                        0
                      • KempRugeLawGroup

                        Fixing Render Blocking Javascript and CSS in the Above-the-fold content

                        We don't have a responsive design site yet, and our mobile site is built through Dudamobile. I know it's not the best, but I'm trying to do whatever we can until we get around to redesigning it. Is there anything I can do about the following Page Speed Insight errors or are they just a function of using Dudamobile? Eliminate render-blocking JavaScript and CSS in above-the-fold content Your page has 3 blocking script resources and 5 blocking CSS resources. This causes a delay in rendering your page.None of the above-the-fold content on your page could be rendered without waiting for the following resources to load. Try to defer or asynchronously load blocking resources, or inline the critical portions of those resources directly in the HTML.Remove render-blocking JavaScript: http://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js http://mobile.dudamobile.com/…ckage.min.js?version=2015-04-02T13:36:04 http://mobile.dudamobile.com/…pts/blogs.js?version=2015-04-02T13:36:04 Optimize CSS Delivery of the following: http://fonts.googleapis.com/…:400|Great+Vibes|Signika:400,300,600,700 http://mobile.dudamobile.com/…ont-pack.css?version=2015-04-02T13:36:04 http://mobile.dudamobile.com/…kage.min.css?version=2015-04-02T13:36:04 http://irp-cdn.multiscreensite.com/kempruge/files/kempruge_0.min.css?v=6 http://irp-cdn.multiscreensite.com/…mpruge/files/kempruge_home_0.min.css?v=6 Thanks for any tips, Ruben

                        Web Design | | KempRugeLawGroup
                        0
                      • CliqStudios

                        Reasons Why Our Website Pages Randomly Loads Without Content

                        I know this is not a marketing question but this community is very dev savvy so I'm hoping someone can help me. At random times we're finding that our website pages load without the main body content. The header, footer and navigation loads just fine. If you refresh, it's fine but that's not a solution. Happens on Chrome, IE and Firefox, testing with multiple browser versions Happens across various page types - but seems to be only the main content section/container Happens while on the company network, as well as externally Happens after deleting cookies, temporary internet files and restarting computer We are using a CMS that is virtually unheard of - Bridgeline/Iapps Codebase is .net Our IT/Dev group keeps pushing back, blaming it on cookies or Chrome plugins because they apparently are unable to "recreate the problem". This has been going on for months and it's a terrible experience for the user to have. It's also not great when landing PPC visitors on pages that load with no content. If anyone has ideas as to why this may be happening I would really appreciate it. I'm not sure if links are allowed, by today the issue happened on this page serversdirect.com/dm/geek-biz Linking to an image example below knEUzqd

                        Web Design | | CliqStudios
                        0
                      • Kingalan1

                        Lots of Listing Pages with Thin Content on Real Estate Web Site-Best to Set them to No-Index?

                        Greetings Moz Community: As a commercial real estate broker in Manhattan I run a web site with over 600 pages. Basically the pages are organized in the following categories: 1. Neighborhoods (Example:http://www.nyc-officespace-leader.com/neighborhoods/midtown-manhattan)  25 PAGES Low bounce rate 2. Types of Space (Example:http://www.nyc-officespace-leader.com/commercial-space/loft-space) 
                        15 PAGES Low bounce rate. 3. Blog (Example:http://www.nyc-officespace-leader.com/blog/how-long-does-leasing-process-take 
                        30 PAGES Medium/high bounce rate 4. Services (Example:http://www.nyc-officespace-leader.com/brokerage-services/relocate-to-new-office-space)  High bounce rate
                        3 PAGES 5. About Us (Example:http://www.nyc-officespace-leader.com/about-us/what-we-do
                        4 PAGES High bounce rate 6. Listings (Example:http://www.nyc-officespace-leader.com/listings/305-fifth-avenue-office-suite-1340sf)
                        300 PAGES High bounce rate (65%), thin content 7. Buildings (Example:http://www.nyc-officespace-leader.com/928-broadway
                        300 PAGES  Very high bounce rate (exceeding 75%) Most of the listing pages do not have more than 100 words.  My SEO firm is advising me to set them "No-Index, Follow". They believe the thin content could be hurting me. Is this an acceptable strategy? I am concerned that when Google detects 300 pages set to "No-Follow" they could interpret this as the site seeking to hide something and penalize us. Also, the building pages have a low click thru rate. Would it make sense to set them to "No-Follow" as well? Basically, would it increase authority in Google's eyes if we set pages that have thin content and/or low click thru rates to "No-Follow"? Any harm in doing this for about half the pages on the site? I might add that while I don't suffer from any manual penalty volume has gone down substantially in the last month. We upgraded the site in early June and somehow 175 pages were submitted to Google  that should not have been indexed. A removal request has been made for those pages. Prior to that we were hit by Panda in April 2012 with search volume dropping from about 7,000 per month to 3,000 per month. Volume had increased back to 4,500 by April this year only to start tanking again. It was down to 3,600 in June. About 30 toxic links were removed in late April and a disavow file was submitted with Google in late April for removal of links from 80 toxic domains. Thanks in advance for your responses!! Alan

                        Web Design | | Kingalan1
                        0
                      • Atlanta-SMO

                        ECWID How to fix Duplicate page content and external link issue

                        I am working on a site that has a HUGE number of duplicate pages due to ECWID ecommerce platform.  The site is built with Joomla!  How can I rectify this situation?  The pages also show up as "external " links on crawls...  Is it the ECWID platform?  I have never worked on a site that uses this. Here is an example of a page with the issue (there are 6280 issues) URL: http://www.metroboltmi.com/shop-spare-parts?Itemid=218&option=com_rokecwid&view=ecwid&ecwid_category_id=3560081

                        Web Design | | Atlanta-SMO
                        0
                      • netbuilder

                        Html 5 main and secondary navigation for SEO best performances

                        I am building a website which will have a main navigation related to the site and each link of the main navigation will have a secondary navigation. We do not want to use a megamenu style navigation. I will try to explain it with a example: Let's start with an example for a computer store "My PC Store", the Main Navigation would be: Desktop PC's Notebook & Tablets
                        Multimedia
                        When clicking on the "Notebook & Tablets" the user is directed to the page domain.com/notebook-tablet.html and on this page the secondary navigation appears: Laptop Netbook Tablets / iPad I am confused on how I should organize the semantic navigation for best SEO performances and I need advice / suggestions. I thought about 2 different ways to do it but which one is more appropriate in terms of SEO? PROPOSITION A Home Page: <header> My PC Store <nav> Desktop PC's Notebook & Tablets Multimedia </nav> </header> Sub-Page (Notebook & Tablets): <nav>(or <aside>?) Desktop PC's Notebook & Tablets Multimedia </aside> </nav> <header> Notebook & Tablets <nav> Laptop Netbook Tablets / iPad </nav> </header> As you notice on the home page the Main Site Navigation is included in the <header>while it is not in the sub-pages. PROPOSITION B Home Page: <header> My PC Store <nav> Desktop PC's Notebook & Tablets Multimedia </nav> </header> Sub-Page (Notebook & Tablets): <header> Notebook & Tablets <nav> Desktop PC's Notebook & Tablets Multimedia </nav> # Notebook & Tablets * Laptop Netbook Tablets / iPad </header> The main navgation remains always in the <header>(home page / sub-pages) of all page. I need suggestions... How would you guys organize the nav ? </header> </header>

                        Web Design | | netbuilder
                        0
                      • markadoi84

                        URLs with Hashtags - Does Google Index Them?

                        Hi there, I have a potential issue with a site whereby all pages are dynamically populated using Javascript.  Thus, an example of an URL on their site would be www.example.com/#!/category/product. I have read lots of conflicting information on the web - some says Google will ignore everything after the hashtag; other people say that Google will now index everything after the hashtag. Does anybody have any conclusive information about this?  Any links to Google or Matt Cutts as confirmation would be brilliant. P.S. I am aware about the potential issue of duplicate content, but I can assure you that has been dealt with.  I am only concerned about whether Google will index full URLs that contain hashtags. Thanks all! Mark

                        Web Design | | markadoi84
                        0

                      Get started with Moz Pro!

                      Unlock the power of advanced SEO tools and data-driven insights.

                      Start my free trial
                      Products
                      • Moz Pro
                      • Moz Local
                      • Moz API
                      • Moz Data
                      • STAT
                      • Product Updates
                      Moz Solutions
                      • SMB Solutions
                      • Agency Solutions
                      • Enterprise Solutions
                      • Digital Marketers
                      Free SEO Tools
                      • Domain Authority Checker
                      • Link Explorer
                      • Keyword Explorer
                      • Competitive Research
                      • Brand Authority Checker
                      • Local Citation Checker
                      • MozBar Extension
                      • MozCast
                      Resources
                      • Blog
                      • SEO Learning Center
                      • Help Hub
                      • Beginner's Guide to SEO
                      • How-to Guides
                      • Moz Academy
                      • API Docs
                      About Moz
                      • About
                      • Team
                      • Careers
                      • Contact
                      Why Moz
                      • Case Studies
                      • Testimonials
                      Get Involved
                      • Become an Affiliate
                      • MozCon
                      • Webinars
                      • Practical Marketer Series
                      • MozPod
                      Connect with us

                      Contact the Help team

                      Join our newsletter
                      Moz logo
                      © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                      • Accessibility
                      • Terms of Use
                      • Privacy

                      Looks like your connection to Moz was lost, please wait while we try to reconnect.