Duplicate content issue with trailing / ?
-
Hi ,I did a SEOmoz Crawl Test and found most pages show twice, for example:
A: www.website.com/index.php/dog/walk
B: www.website.com/index.php/dog/walk/
I've checked Google Analytics and 90% of organic search traffic arrives on the URLs with the trailing slash (B).
Question 1: Can I assume I've a duplicate content problem?
Question 2: Is it best to do 301 redirects from the 'non trailing slash' pages to the 'trailing slash pages'?
Question 3: For some reason every web page has a '/index.php' in it (see A&B) above. No idea why. Should it be a SEO concern?
Kind regards and thank you in advance
Nigel
-
Hi Nigel
You only need to 301 one of the pages, 301 is indicating a permanent move, so in the case you outlined above,
I would 301, A to B the decisions to use B was based soly off the value of the url you indicated. If for any reason you prefer the url's not use trailing slash then use A.
It also would not hurt to add a canonical tag to B
To be clear here, whether you use
website.com/index.php/dog/walk
or
website.com/index.php/dog/walk/
Does not matter as far as SEO is concerned, I would make my decision based off of which url has the highest position in Google, and be consistent with this method throughout my site.
Hope that helps,
-
Hi Irving
Thank you for your reply. You mention a good point regarding the sitemap.xml!
If I was to 301redirect pages A & B to a new page eg www.website.com/dog/walk/ then how would I also canonical A & B to the new page?
Surely once I have 301'd the A & B pages will be dead and redirecting traffic to the new page.
Kind regard and my apologies for any confusion.
Nigel
-
Yes, index.php should never show so 301 that plus the trailing slash to remove it
Ddefinitely canonical all of the pages to have the URL without the trailing slash
Make sure your sitemap xml files and internal linking structure does not have the trailing slash. if they do,, then fix them to reflect the proper URL
-
Thank you Highland & Donford.
Re my 3rd question, can I just clarify, should I now 301 redirect both A & B URLs to a new URL say www.website/com/dog/walk ?
Many thanks!
-
Question 1: Can I assume I've a duplicate content problem?
-YesQuestion 2: Is it best to do 301 redirects from the 'non trailing slash' pages to the 'trailing slash pages'?
-Yes 301 is best, barring that use rel="canonical" on the page you want to indexQuestion 3: For some reason every web page has a '/index.php' in it (see A&B) above. No idea why. Should it be a SEO concern?
-Yes, this is a concern, use the same method to deal with the problem. Directories on the server side are usually assumed to have an index, if not the server can choose what to display, this can be very bad sometimes. As such most CMS content management systems fix the problem by generating content for the index.php or .html pages. However, there can be duplicate content issues since there are 2 urls with the same content, use 301 to get rid of the index.php at directory levels, or use canonical tags.
Hope that helps,
Don
-
1. Google can generally tell the difference between pages that have syntactically similar URLs but it's considered a best practice to not make any engine do any guesswork whenever possible.
2. I would 301 one version just for uniformity but you should be fine as-is right now.
3. There's nothing wrong with that being in the URL. Google sees it as part of the URL and nothing more. I don't consider it aesthetic or user friendly but that's a different matter.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content problem
Hi, i work in joomla and my site is www.in2town.co.uk I have been looking at moz tools and it is showing i have over 600 pages of duplicate content. The problem is shown below and i am not sure how to solve this, any help would be great, | Benidorm News http://www.in2town.co.uk/benidorm-news/Page-2 50 1 0 In2town http://www.in2town.co.uk/blog/In2town/Page-102 50 23 3 In2town http://www.in2town.co.uk/blog/In2town/Page-103 50 23 3 In2town http://www.in2town.co.uk/blog/In2town/Page-104 9 23 3 In2town http://www.in2town.co.uk/blog/In2town/Page-106 28 23 3 In2town http://www.in2town.co.uk/blog/In2town/Page-11 50 22 3 In2town http://www.in2town.co.uk/blog/In2town/Page-112 50 23 3 In2town http://www.in2town.co.uk/blog/In2town/Page-114 45 23 3 In2town http://www.in2town.co.uk/blog/In2town/Page-115 50 23 3 In2town http://www.in2town.co.uk/blog/In2town/Page-116 50 23 3 In2town http://www.in2town.co.uk/blog/In2town/Page-12 50 22 3 In2town http://www.in2town.co.uk/blog/In2town/Page-120 50 23 3 In2town http://www.in2town.co.uk/blog/In2town/Page-123 50 23 3 In2town http://www.in2town.co.uk/blog/In2town/Page-13 50 22 3 In2town http://www.in2town.co.uk/blog/In2town/Page-130 50 23 3 In2town http://www.in2town.co.uk/blog/In2town/Page-131 50 22 3 In2town http://www.in2town.co.uk/blog/In2town/Page-132 31 22 3 In2town http://www.in2town.co.uk/blog/In2town/Page-140 4 18 1 In2town http://www.in2town.co.uk/blog/In2town/Page-141 50 1 0 In2town http://www.in2town.co.uk/blog/In2town/Page-21 10 18 1 In2town http://www.in2town.co.uk/blog/In2town/Page-22 50 18 1 In2town http://www.in2town.co.uk/blog/In2town/Page-23 50 18 1 In2town http://www.in2town.co.uk/blog/In2town/Page-26 50 18 1 In2town http://www.in2town.co.uk/blog/In2town/Page-271 50 18 1 In2town http://www.in2town.co.uk/blog/In2town/Page-274 50 18 1 In2town http://www.in2town.co.uk/blog/In2town/Page-277 50 21 2 In2town http://www.in2town.co.uk/blog/In2town/Page-28 50 21 2 In2town http://www.in2town.co.uk/blog/In2town/Page-29 50 18 1 In2town http://www.in2town.co.uk/blog/In2town/Page-310 50 1 0 In2town http://www.in2town.co.uk/blog/In2town/Page-341 21 1 0 In2town http://www.in2town.co.uk/blog/In2town/Page-342 4 1 0 In2town http://www.in2town.co.uk/blog/In2town/Page-343 50 1 0 In2town http://www.in2town.co.uk/blog/In2town/Page-345 1 1 0 In2town http://www.in2town.co.uk/blog/In2town/Page-346 50 1 0 In2town http://www.in2town.co.uk/blog/In2town/Page-348 50 1 0 In2town http://www.in2town.co.uk/blog/In2town/Page-349 50 1 0 In2town http://www.in2town.co.uk/blog/In2town/Page-350 50 16 0 In2town http://www.in2town.co.uk/blog/In2town/Page-351 50 19 1 In2town http://www.in2town.co.uk/blog/In2town/Page-82 24 1 0 In2town http://www.in2town.co.uk/blog/in2town 50 20 1 In2town http://www.in2town.co.uk/blog/in2town/Page-10 50 23 3 In2town http://www.in2town.co.uk/blog/in2town/Page-100 50 22 3 In2town http://www.in2town.co.uk/blog/in2town/Page-101 50 22 3 In2town http://www.in2town.co.uk/blog/in2town/Page-105 50 22 3 In2town http://www.in2town.co.uk/blog/in2town/Page-107 50 22 3 In2town http://www.in2town.co.uk/blog/in2town/Page-108 50 22 3 In2town http://www.in2town.co.uk/blog/in2town/Page-109 50 22 3 In2town http://www.in2town.co.uk/blog/in2town/Page-110 50 22 3 In2town http://www.in2town.co.uk/blog/in2town/Page-111 50 22 3 In2town http://www.in2town.co.uk/blog/in2town/Page-113 |
Technical SEO | | ClaireH-1848860 -
Content and url duplication?
One of the campaign tools flags one of my clients sites as having lots of duplicates. This is true in the sense the content is sort of boiler plate but with the different countries wording changed. The is same with the urls but they are different in the sense a couple of words have changed in the url`s. So its not the case of a cms or server issue as this seomoz advises. It doesnt need 301`s! Thing is in the niche, freight, transport operators, shipping, I can see many other sites doing the same thing and those sites have lots of similar pages ranking very well. In fact one site has over 300 keywords ranked on page 1-2, but it is a large site with an 12yo domain, which clearly helps. Of course having every page content unique is important, however, i suppose it is better than copy n paste from other sites. So its unique in that sense. Im hoping to convince the site owner to change the content over time for every country. A long process. My biggest problem for understanding duplication issues is that every tabloid or broadsheet media website would be canned from google as quite often they scrape Reuters or re-publish standard press releases on their sites as newsworthy content. So i have great doubt that there is a penalty for it. You only have to look and you can see media sites duplication everywhere, everyday, but they get ranked. I just think that google dont rank the worst cases of spammy duplication. They still index though I notice. So considering the business niche has very much the same content layout replicated content, which rank well, is this duplicate flag such a great worry? Many businesses sell the same service to many locations and its virtually impossible to re write the services in a dozen or so different ways.
Technical SEO | | xtopher660 -
Squarespace Duplicate Content Issues
My site is built through squarespace and when I ran the campaign in SEOmoz...its come up with all these errors saying duplicate content and duplicate page title for my blog portion. I've heard that canonical tags help with this but with squarespace its hard to add code to page level...only site wide is possible. Was curious if there's someone experienced in squarespace and SEO out there that can give some suggestions on how to resolve this problem? thanks
Technical SEO | | cmjolley0 -
Does turning website content into PDFs for document sharing sites cause duplicate content?
Website content is 9 tutorials published to unique urls with a contents page linking to each lesson. If I make a PDF version for distribution of document sharing websites, will it create a duplicate content issue? The objective is to get a half decent link, traffic to supplementary opt-in downloads.
Technical SEO | | designquotes0 -
Duplicate Page Content
I've got several pages of similar products that google has listed as duplicate content. I have them all set up with rel="prev" and rel="next tags telling google that they are part of a group but they've still got them listed as duplicates. Is there something else I should do for these pages or is that just a short falling of googles webmaster tools? One of the pages: http://www.jaaronwoodcountertops.com/wood-countertop-gallery/walnut-countertop-9.html
Technical SEO | | JAARON0 -
Help removing duplicate content from the index?
Last week, after a significant drop in traffic, I noticed a subdomain in the index with duplicate content. The main site and subdomain can be found below. http://mobile17.com http://232315.mobile17.com/ I've 301'd everything on the subdomain to the appropriate location on the main site. Problem is, site: searches show me that if the subdomain content is being deindexed, it's happening really slowly. Traffic is still down about 50% in the last week or so... what's the best way to tackle this issue moving forward?
Technical SEO | | ccorlando0 -
How to see a theme ‘/wp-content/themes/’
HI I'm still plugging away at getting to grips with my companies personalized blog. I've been trying for the past two days to upload a theme to my own test Wordpress blog, in order to correct a bug in the companies theme that makes formatting in the Post disappear. The code in the themes CSS file seems to be fine. Anyhow what I assumed would be a simple step has given me hours of hassle. I have finally got to the point of uploading an unzipped version of the theme intot ‘/wp-content/themes/’. Now try as I might my Wordpress admin is completely blind to the fact. Any attempt at using the Upload facility (which is what I attempted many hours ago) fails. There seems to be no place to say, look out there at my directory - a new original theme - unzipped and ready to go. Am I missing something very obvious?
Technical SEO | | catherine-2793880 -
Duplicate content
I am getting flagged for duplicate content, SEOmoz is flagging the following as duplicate: www.adgenerator.co.uk/ www.adgenerator.co.uk/index.asp These are obviously meant to be the same path so what measures do I take to let the SE's know that these are to be considered the same page. I have used the canonical meta tag on the Index.asp page.
Technical SEO | | IPIM0