Duplicate content issue with trailing / ?
-
Hi ,I did a SEOmoz Crawl Test and found most pages show twice, for example:
A: www.website.com/index.php/dog/walk
B: www.website.com/index.php/dog/walk/
I've checked Google Analytics and 90% of organic search traffic arrives on the URLs with the trailing slash (B).
Question 1: Can I assume I've a duplicate content problem?
Question 2: Is it best to do 301 redirects from the 'non trailing slash' pages to the 'trailing slash pages'?
Question 3: For some reason every web page has a '/index.php' in it (see A&B) above. No idea why. Should it be a SEO concern?
Kind regards and thank you in advance
Nigel
-
Hi Nigel
You only need to 301 one of the pages, 301 is indicating a permanent move, so in the case you outlined above,
I would 301, A to B the decisions to use B was based soly off the value of the url you indicated. If for any reason you prefer the url's not use trailing slash then use A.
It also would not hurt to add a canonical tag to B
To be clear here, whether you use
website.com/index.php/dog/walk
or
website.com/index.php/dog/walk/
Does not matter as far as SEO is concerned, I would make my decision based off of which url has the highest position in Google, and be consistent with this method throughout my site.
Hope that helps,
-
Hi Irving
Thank you for your reply. You mention a good point regarding the sitemap.xml!
If I was to 301redirect pages A & B to a new page eg www.website.com/dog/walk/ then how would I also canonical A & B to the new page?
Surely once I have 301'd the A & B pages will be dead and redirecting traffic to the new page.
Kind regard and my apologies for any confusion.
Nigel
-
Yes, index.php should never show so 301 that plus the trailing slash to remove it
Ddefinitely canonical all of the pages to have the URL without the trailing slash
Make sure your sitemap xml files and internal linking structure does not have the trailing slash. if they do,, then fix them to reflect the proper URL
-
Thank you Highland & Donford.
Re my 3rd question, can I just clarify, should I now 301 redirect both A & B URLs to a new URL say www.website/com/dog/walk ?
Many thanks!
-
Question 1: Can I assume I've a duplicate content problem?
-YesQuestion 2: Is it best to do 301 redirects from the 'non trailing slash' pages to the 'trailing slash pages'?
-Yes 301 is best, barring that use rel="canonical" on the page you want to indexQuestion 3: For some reason every web page has a '/index.php' in it (see A&B) above. No idea why. Should it be a SEO concern?
-Yes, this is a concern, use the same method to deal with the problem. Directories on the server side are usually assumed to have an index, if not the server can choose what to display, this can be very bad sometimes. As such most CMS content management systems fix the problem by generating content for the index.php or .html pages. However, there can be duplicate content issues since there are 2 urls with the same content, use 301 to get rid of the index.php at directory levels, or use canonical tags.
Hope that helps,
Don
-
1. Google can generally tell the difference between pages that have syntactically similar URLs but it's considered a best practice to not make any engine do any guesswork whenever possible.
2. I would 301 one version just for uniformity but you should be fine as-is right now.
3. There's nothing wrong with that being in the URL. Google sees it as part of the URL and nothing more. I don't consider it aesthetic or user friendly but that's a different matter.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Localized domains and duplicate content
Hey guys, In my company we are launching a new website and there's an issue it's been bothering me for a while. I'm sure you guys can help me out. I already have a website, let's say ABC.com I'm preparing a localized version of that website for the uk so we'll launch ABC.co.uk Basically the websites are going to be exactly the same with the difference of the homepage. They have a slightly different proposition. Using GeoIP I will redirect the UK traffic to ABC.co.uk and the rest of the traffic will still visit .com website. May google penalize this? The site itself it will be almost the same but the homepage. This may count as duplicate content even if I'm geo-targeting different regions so they will never overlap. Thanks in advance for you advice
Technical SEO | | fabrizzio0 -
How to protect against duplicate content?
I just discovered that my company's 'dev website' (which mirrors our actual website, but which is where we add content before we put new content to our actual website) is being indexed by Google. My first thought is that I should add a rel=canonical tag to the actual website, so that Google knows that this duplicate content from the dev site is to be ignored. Is that the right move? Are there other things I should do? Thanks!
Technical SEO | | williammarlow0 -
Lots of duplicate content warnings
I have a site that says that I have 2,500 warnings. It is a real estate website and of course we use feeds. it says I have a lot of duplicate content. One thing is a page called "Request an appointment" and that is a url for each listing. Since there are 800 listings on my site. How could I solve this problem so that this doesn't show up as duplicate content since I use the same "Request an Appointment" verbeage on each of those? I guess my developer who used php to do it, created a dedicated url to each. Any help would be greatly appreciated.
Technical SEO | | SeaC0 -
Local Search | Website Issue with Duplicate Content (97 pages)
Hi SEOmoz community. I have a unique situation where I’m evaluating a website that is trying to optimize better for local search and targeting 97 surrounding towns in his geographical location. What is unique about this situation is that he is ranking on the 1st and 2nd pages of the SERPs for his targeted keywords, has duplicate content on 97 pages to his site, and the search engines are still ranking the website. I ran the website’s url through SEOmoz’s Crawl Test Tool and it verified that it has duplicate content on 97 pages and has too many links (97) per page. Summary: Website has 97 duplicate pages representing each town, with each individual page listing and repeating all of the 97 surrounding towns, and each town is a link to a duplicate page. Question: I know eventually the site will not get indexed by the Search Engines and not sure the best way to resolve this problem – any advice?
Technical SEO | | ToddSEOBoston0 -
What's the best way to solve this sites duplicate content issues?
Hi, The site is www.expressgolf.co.uk and is an e-commerce website with lots of categories and brands. I'm trying to achieve one single unique URL for each category / brand page to avoid duplicate content and to get the correct URL's indexed. Currently it looks like this... Main URL http://www.expressgolf.co.uk/shop/clothing/galvin-green Different Versions http://www.expressgolf.co.uk/shop/clothing/galvin-green/ http://www.expressgolf.co.uk/shop/clothing/galvin-green/1 http://www.expressgolf.co.uk/shop/clothing/galvin-green/2 http://www.expressgolf.co.uk/shop/clothing/galvin-green/3 http://www.expressgolf.co.uk/shop/clothing/galvin-green/4 http://www.expressgolf.co.uk/shop/clothing/galvin-green/all http://www.expressgolf.co.uk/shop/clothing/galvin-green/1/ http://www.expressgolf.co.uk/shop/clothing/galvin-green/2/ http://www.expressgolf.co.uk/shop/clothing/galvin-green/3/ http://www.expressgolf.co.uk/shop/clothing/galvin-green/4/ http://www.expressgolf.co.uk/shop/clothing/galvin-green/all/ Firstly, what is the best course of action to make all versions point to the main URL and keep them from being indexed - Canonical Tag, NOINDEX or block them in robots? Secondly, do I just need to 301 the (/) from all URL's to the non (/) URL's ? I'm sure this question has been answered but I was having trouble coming to a solution for this one site. Cheers, Paul
Technical SEO | | paulmalin0 -
A site I am working with has multiple duplicate content issues.
A reasonably large ecommerce site I am working with has multiple duplicate content issues. On 4 or 5 keyword domains related to site content the owners simply duplicated the home page with category links pushing visitors to the category pages of the main site. There was no canonical URL instruction, so have set preferred url via webmaster tools but now need to code this into the website itself. For a reasonably large ecommerce site, how would you approach that particular nest of troubles. That's even before we get to grips with the on page duplication and wrong keywords!
Technical SEO | | SkiBum0 -
Duplicate content
Greetings! I have inherited a problem that I am not sure how to fix. The website I am working on had a 302 redirect from its original home url (with all the link juice) to a newly designed page (with no real link juice). When the 302 redirect was removed, a duplicate content problem remained, since the new page had already been indexed by google. What is the best way to handle duplicate content? Thanks!
Technical SEO | | shedontdiet0 -
Duplicate Content from Google URL Builder
Hello to the SEOmoz community! I am new to SEOmoz, SEO implementation, and the community and recently set up a campaign on one of the sites I managed. I was surprised at the amount of duplicate content that showed up as errors and when I took a look in deeper, the majority of errors were caused by pages on the root domain I put through Google Analytics URL Builder. After this, I went into webmaster tools and changed the parameter handling to ignore all of the tags the URL Builder adds to the end of the domain. SEOmoz recently recrawled my site and the errors being caused by the URL Builder are still being shown as duplicates. Any suggestions on what to do?
Technical SEO | | joshuaopinion0