Duplicate content issue with trailing / ?
-
Hi ,I did a SEOmoz Crawl Test and found most pages show twice, for example:
A: www.website.com/index.php/dog/walk
B: www.website.com/index.php/dog/walk/
I've checked Google Analytics and 90% of organic search traffic arrives on the URLs with the trailing slash (B).
Question 1: Can I assume I've a duplicate content problem?
Question 2: Is it best to do 301 redirects from the 'non trailing slash' pages to the 'trailing slash pages'?
Question 3: For some reason every web page has a '/index.php' in it (see A&B) above. No idea why. Should it be a SEO concern?
Kind regards and thank you in advance
Nigel
-
Hi Nigel
You only need to 301 one of the pages, 301 is indicating a permanent move, so in the case you outlined above,
I would 301, A to B the decisions to use B was based soly off the value of the url you indicated. If for any reason you prefer the url's not use trailing slash then use A.
It also would not hurt to add a canonical tag to B
To be clear here, whether you use
website.com/index.php/dog/walk
or
website.com/index.php/dog/walk/
Does not matter as far as SEO is concerned, I would make my decision based off of which url has the highest position in Google, and be consistent with this method throughout my site.
Hope that helps,
-
Hi Irving
Thank you for your reply. You mention a good point regarding the sitemap.xml!
If I was to 301redirect pages A & B to a new page eg www.website.com/dog/walk/ then how would I also canonical A & B to the new page?
Surely once I have 301'd the A & B pages will be dead and redirecting traffic to the new page.
Kind regard and my apologies for any confusion.
Nigel
-
Yes, index.php should never show so 301 that plus the trailing slash to remove it
Ddefinitely canonical all of the pages to have the URL without the trailing slash
Make sure your sitemap xml files and internal linking structure does not have the trailing slash. if they do,, then fix them to reflect the proper URL
-
Thank you Highland & Donford.
Re my 3rd question, can I just clarify, should I now 301 redirect both A & B URLs to a new URL say www.website/com/dog/walk ?
Many thanks!
-
Question 1: Can I assume I've a duplicate content problem?
-YesQuestion 2: Is it best to do 301 redirects from the 'non trailing slash' pages to the 'trailing slash pages'?
-Yes 301 is best, barring that use rel="canonical" on the page you want to indexQuestion 3: For some reason every web page has a '/index.php' in it (see A&B) above. No idea why. Should it be a SEO concern?
-Yes, this is a concern, use the same method to deal with the problem. Directories on the server side are usually assumed to have an index, if not the server can choose what to display, this can be very bad sometimes. As such most CMS content management systems fix the problem by generating content for the index.php or .html pages. However, there can be duplicate content issues since there are 2 urls with the same content, use 301 to get rid of the index.php at directory levels, or use canonical tags.
Hope that helps,
Don
-
1. Google can generally tell the difference between pages that have syntactically similar URLs but it's considered a best practice to not make any engine do any guesswork whenever possible.
2. I would 301 one version just for uniformity but you should be fine as-is right now.
3. There's nothing wrong with that being in the URL. Google sees it as part of the URL and nothing more. I don't consider it aesthetic or user friendly but that's a different matter.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Managing Zendesk content, specifically redirecting/retiring content?
We've been using Zendesk to manage support content, and have old/duplicate articles that we'd like to redirect. However, Zendesk doesn't seem to have a solution for this, and the suggestions we've found (some hacky JS) have not worked. I'd like for us to not just delete/hide these articles. Has anyone else successfully navigated retiring/redirecting Zendesk content in an SEO-friendly fashion?
Technical SEO | | KMStrava0 -
How to fix HTTP/HTTPS duplicate content
I recently installed an SSL certificate on the site: https://libertywholesalesupply.com Moz is now reading thousands of duplicate content pages because it is reading both http and https. I set up the configuration in Magento to auto-redirect the base URL, created a permanent redirect for the URL in the SEO settings, and adjusted the canonical settings. What am I missing??
Technical SEO | | adamxj20 -
WordPress Duplicate Content Caused By Categories
Hello, We have a wordpress blog that has around 250 categories. Due to our platform we have a hierarchy structure for 3 separate stores. For example iPhone > Apps > Books. Placing a blog post in the books category automatically places it into iPhone and iPhone/Apps category, causing 3 instances of any blog post in this category. Is this an issue? I have seen 2 schools of thought on categories, 1 index follow and 2 noindex follow. I know some of our categories get indexed, but with so many, maybe it is better to noindex them. We also considered reducing our categories to 10 to 12 and use tags to provide the indexed site navigation as follows: Reviews (category) iPhone Book App, iPhone App Store (tags) but this seems a little redundant? Anyone want to take this on? thank you Mike
Technical SEO | | crazymikesapps10 -
Localized domains and duplicate content
Hey guys, In my company we are launching a new website and there's an issue it's been bothering me for a while. I'm sure you guys can help me out. I already have a website, let's say ABC.com I'm preparing a localized version of that website for the uk so we'll launch ABC.co.uk Basically the websites are going to be exactly the same with the difference of the homepage. They have a slightly different proposition. Using GeoIP I will redirect the UK traffic to ABC.co.uk and the rest of the traffic will still visit .com website. May google penalize this? The site itself it will be almost the same but the homepage. This may count as duplicate content even if I'm geo-targeting different regions so they will never overlap. Thanks in advance for you advice
Technical SEO | | fabrizzio0 -
How do I Address Low Quality/Duplicate Content Issue for a Job portal?
Hi, I want to optimize my job portal for maximum search traffic. Problems Duplicate content- The portal takes jobs from other portals/blogs and posts on our site. Sometimes employers provide the same job posting to multiple portals and we are not allowed to change it resulting in duplicate content Empty Content Pages- We have a lot of pages which can be reached via filtering for multiple options. Like IT jobs in New York. If there are no IT jobs posted in New York, then it's a blank page with little or no content Repeated Content- When we have job postings, we have about the company information on each job listing page. If a company has 1000 jobs listed with us, that means 1000 pages have the exact same about the company wording Solutions Implemented Rel=prev and next. We have implemented this for pagination. We also have self referencing canonical tags on each page. Even if they are filtered with additional parameters, our system strips of the parameters and shows the correct URL all the time for both rel=prev and next as well as self canonical tags For duplicate content- Due to the volume of the job listings that come each day, it's impossible to create unique content for each. We try to make the initial paragraph (at least 130 characters) unique. However, we use a template system for each jobs. So a similar pattern can be detected after even 10 or 15 jobs. Sometimes we also take the wordy job descriptions and convert them into bullet points. If bullet points already available, we take only a few bullet points and try to re-shuffle them at times Can anyone provide me additional pointers to improve my site in terms of on-page SEO/technical SEO? Any help would be much appreciated. We are also thinking of no-indexing or deleting old jobs once they cross X number of days. Do you think this would be a smart strategy? Should I No-index empty listing pages as well? Thank you.
Technical SEO | | jombay3 -
Over 700+ duplicate content pages -- help!
I just signed up for SEO Moz pro for my site. The initial report came back with over 700+ duplicate content pages. My problem is that while I can see why some of the content is duplicated on some of the pages I have no idea why it's coming back as duplicated. Is there a tutorial for a novie on how to read the duplicate content report and what steps to take? It's an e-commerce website and there is some repetitive content on all the product pages like our "satisfaction guaranteed" text and the fabric material... and not much other text. There's not a unique product description because an image speaks for itself. Could this be causing the problem? I have lots of URLs with over 50+ duplicates. Thx for any help.
Technical SEO | | Santaur0 -
404's and duplicate content.
I have real estate based websites that add new pages when new listings are added to the market and then deletes pages when the property is sold. My concern is that there are a significant amount of 404's created and the listing pages that are added are going to be the same as others in my market who use the same IDX provider. I can go with a different IDX provider that uses IFrame which doesn't create new pages but I used a IFrame before and my time on site was 3min w/ 2.5 pgs per visit and now it's 7.5 pg/visit with 6+min on the site. The new pages create new content daily so is fresh content and better on site metrics (with the 404's) better or less 404's, no dup content and shorter onsite metrics better? Any thoughts on this issue? Any advice would be appreciated
Technical SEO | | AnthonyLasVegas0 -
Are recipes excluded from duplicate content?
Does anyone know how recipes are treated by search engines? For example, I know press releases are expected to have lots of duplicates out there so they aren't penalized. Does anyone know if recipes are treated the same way. For example, if you Google "three cheese beef pasta shells" you get the first two results with identical content.
Technical SEO | | RiseSEO0