Duplicate content warning: Same page but different urls???
-
Hi guys i have a friend of mine who has a site i noticed once tested with moz that there are 80 duplicate content warnings, for instance
Page 1 is http://yourdigitalfile.com/signing-documents.html
the warning page is http://www.yourdigitalfile.com/signing-documents.html
another example
Page 1 http://www.yourdigitalfile.com/
same second page http://yourdigitalfile.com
i noticed that the whole website is like the nealry every page has another version in a different url?, any ideas why they dev would do this, also the pages that have received the warnings are not redirected to the newer pages you can go to either one???
thanks very much
-
Thanks Tim. Do you have any examples of what those problems might be? With such a large catalog managing those rel canonical tags will be difficult (I don't even know if the store allows them, it's a hosted store solution and little code customization is allowed).
-
Hi there AspenFasteners, in this instance rather than a .HTAccess rule I would suggest applying a rel canonical tag which points to the page you deem as the original master source.
Using the robots to try and hide things could potentially cause you more issues as your categories may struggle to be indexed correctly.
-
We have a similar problem, but much more complex to handle as we have a massive catalog of 80,000 products and growing.
The problem occurs legitimately because our catalog is so large that we offer different navigation paths to the same content.
http://www.aspenfasteners.com/Self-Tapping-Sheet-Metal-s/8314.htm
http://www.aspenfasteners.com/Self-Tapping-Sheet-Metal-s/8315.htm
(If you look at the "You are here" breadcrumb trail, you will see the subtle differences in the navigation paths, with 8314.htm, the user went through Home > Screws, with 8315.htm, via Home > Security Fasteners > Screws).
Our hosted web store does not offer us htaccess, so I am thinking of excluding the redundant navigation points via robots.txt.
My question: is there any reason NOT to do this?
-
Oh ok
The only reason i was thinking it is duplicate content is the warnings i got on the moz crawl, see below.
75 Duplicate Page Content
6 4xx Client Error
5 Duplicate Page Title
44 Missing Meta Description Tag
5 Title Element is Too Short
I have found over 80 typos, grammatical errors, punctuation errors and incorrect information which was leading me to believe the quality of the work and their attention to detail was rather bad, which is why i thought this was a possibility.
Thanks again for your time its really appreciated
-
I wouldn't say that they have created two pages, it is just that because you have two versions of the domain and not set a preferred version that you are getting it indexing twice. .HTaccess changes are under the hood of the website and could have simply been an oversight.
-
Hey Tim
Thanks for your answer. It's really weird, other than lazyness on the devs part not to remove old or previous versions of pages?, have you any idea why they would create multiple versions of the same page with different url's?? is there any legit reason like ones severs mobile or something??
Just wondering thanks for replying
-
OK, so in this instance the only issue you have is that you need to choose your preferred start point - www or non www.
I would add a bit of code to your htaccess file to point to your preferred choice. I personally prefer a www. domain. Something like the below would work.
RewriteCond %{HTTP_HOST} ^example.com$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]As your site is already indexed I would also for the time being and as more of a safety measure add canonicals to the pages that point to the www. version of your site.
Also if you have a Google Search Console account, you can select your prefered domain prefix in there. this will again help with your indexation.
Hopefully I have covered most things.
Cheers
Tim
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO - All topic related pages in same directory?
Hey Mozzers, How would you structure the following pages for SEO. The site is a multi-product / multi-topic site, but all pages in this example are based on a single topic - CRM Software: CRM Software product CRM Software related blog post 1 CRM Software related blog post 2 CRM Software related blog post 3 CRM Software downloadable resource 1 CRM Software downloadable resource 2 CRM Software downloadable resource 3 I know building directory pyramids is a bit old hat nowadays, but I still see the odd website organising the above pages, as follows: /crm-software /crm-software/crm-blog-post-1 /crm-software/crm-blog-post-2 /crm-software/crm-blog-post-3 /crm-software/crm-resource-1 /crm-software/crm-resource-2 /crm-software/crm-resource-3 However, I'm more inclined to apply a more logical structure, as follows: /crm-software /blog/crm-blog-post-1 /blog/crm-blog-post-2 /blog/crm-blog-post-3 /resources/crm-resource-1 /resources/crm-resource-2 /resources/crm-resource-3 What would you say is SEO best practice? Thanks!
White Hat / Black Hat SEO | | Zoope0 -
Forcing Google to Crawl a Backlink URL
I was surprised that I couldn't find much info on this topic, considering that Googlebot must crawl a backlink url in order to process a disavow request (ie Penguin recovery and reconsideration requests). My trouble is that we recently received a great backlink from a buried page on a .gov domain and the page has yet to be crawled after 4 months. What is the best way to nudge Googlebot into crawling the url and discovering our link?
White Hat / Black Hat SEO | | Choice0 -
How long we can keep 302 redirection for a webpage url?
Hi Friends, I have a webpage featuring a product. I have created a new domain featuring the same product and the page is under construction. I am planning to do 302 redirection from the new domain to the existing domain for the time being. How long can I keep the 302 redirection from the new domain to existing domain? Is there any fixed time period/ duration that we can keep the 302 redirection for a webpage? I am planning to make few more pages (privacy policy, about us, etc) from the new domain 302 redirected to the existing domain. Is it possible? If so, how long can I keep the same? May I know which redirect is safe to use in this case, 302 or 301 redirect?
White Hat / Black Hat SEO | | zco_seo0 -
Footer links VS Page links - Which one is best?
Hello all 🙂 I was wondering if someone could advise me on a link building question. If you wish to create a couple of landing pages for different locations with anchor text link building etc is it better to have a page like this web site here: http://www.acorncommercial.co.uk/commercial-property/development-sites/ or quick footer links like this web site here?: http://www.robertholmes.co.uk/ (click on quick links at the bottom). I would like to know if there is a difference from an SEO perspective or if they are considered black hat. Your advise would be much appreciated! Yiannis
White Hat / Black Hat SEO | | artdivision0 -
Creating duplicate site for testing purpose. Can it hurt original site
Hello, We are soon going to upgrade the cms to latest version along with new functionlaities - the process may take anywhere from 4 week to 6 weeks. may suggest - we need to work on live server, what we have planned take exact replica of site and move to a test domain, but on live server Block Google, Bing, Yahoo - User-agent: Google Disallow: / , User-agent: Bing Disallow: / User-agent: Yahoo Disallow: / in robots.txt Will upgrade CMS and add functionality - will test the entire structure, check url using screaming frog or xenu and move on to configure the site on original domain The process upgradation and new tools may take 1 - 1.5 month.... Concern is that despite blocking Google, Bing & Yahoo through User agent disallow - can still the url can be crawled by the search engines - if yes - it may hurt the original site as will read on as entire duplicate or is there any alternate way around.. Many thanks
White Hat / Black Hat SEO | | Modi1 -
URL Structure - forward slashes, hyphen separated, query paramters
I am having difficulty evaluating pros and cons of various URL structures with respect to SEO benefits. So I can have the following 1. /for-sale-in-<city>-<someothertext>-<uniqueid>.php
White Hat / Black Hat SEO | | proptiger
So in this case a term like 'for sale in San Francisco' is directly part of the URL. </uniqueid></someothertext></city> 2. /for-sale/<city>/<someothertext>uniqueId
Here 'for sale in San Francisco' is not so direct in the URL, so I think. Also I 'heard' that forward slash URLs are somehow considered as being 'lower down' in the directory structure. </someothertext></city> 3. /for-sale/<city>/<someothertext>/?pid=uniqueId</someothertext></city> someOtherText contains keywords we are targeting. 1. Is there a preference of one format over the other? 2. Does it even matter? 3. someOtherText - does it makes sense to put keywords in the URL for just SEO purposes? I do not per se need someOtherText for functionality.0 -
Include placename in URL, or not?
Hi Mozzers, I'm wondering whether to put placename in URL or not. This is for a hotel so it's very focused on the county. I have loads of sub pages along the lines of www.hotelname.com/short-breaks-somerset www.hotelname.com/eat-out-somerset and so on but I was wondering whether that placename element would help or hinder. For example, may want to rank for short breaks in other searches (not just those seeking short breaks in Somerset) and was wondering whether the somerset bit may actually hinder this in the future. Also noticed Somerset is mentioned in nearly all of the page urls through the site. Perhaps this is a bit spammy and just not neccesary. I can include the address of the hotel on every page anyway. What do you think? Thanks in advance for your help 🙂 Luke
White Hat / Black Hat SEO | | McTaggart0 -
Link Building: Location-specific pages
Hi! I've technically been a member for a few years, but just recently decided to go Pro (and I gotta say, I'm glad I did!). Anyway, as I've been researching and analyzing, one thing I noticed a competitor is doing is creating location-specific pages. For example, they've created a page that has a URL similar to this: www.theirdomain.com/seattle-keyword-phrase They have a few of these for specific cities. They rank well for the city-keyword combo in most cases. Each city-specific page looks the same and the content is close to being the same except that they drop in the "seattle keyword phrase" bit here and there. I noticed that they link to these pages from their site map page, which, if I were to guess, is how SEs are getting to those pages. I've seen this done before on other sites outside my industry too. So my question is, is this good practice or is it something that should be avoided?
White Hat / Black Hat SEO | | AngieHerrera0