Duplicate content issues caused by our CMS

[themegroup]

Hello fellow mozzers,

Our in-house CMS - which is usually good for SEO purposes as it allows all the control over directories, filenames, browser titles etc that prevent unwieldy / meaningless URLs and generic title tags - seems to have got itself into a bit of a tiz when it comes to one of our clients.

We have tried solving the problem to no avail, so I thought I'd throw it open and see if anyone has a soultion, or whether it's just a fault in our CMS.

Basically, the SEs are indexing two identical pages, one ending with a / and the other ending /index.php, for one of our sites (www.signature-care-homes.co.uk).

We have gone through the site and made sure the links all point to just one of these, and have done the same for off-site links, but there is still the duplicate content issue of both versions getting indexed.

We also set up an htaccess file to redirect to the chosen version, but to no avail, and we're not sure canonical will work for this issue as / pages should redirect to /index.php anyway - and that's we can't work out. We have set the access file to point to index.php, and that should be what should be happening anyway, but it isn't.

Is there an alternative way of telling the SE's to only look at one of these two versions?

Also, we are currently rewriting the content and changing the structure - will this change the situation we find ourselves in?

[jsturgeon]

Hi Nick,

Given that you have tried all of the above, I recommend cutting off the search engines at the source, in your Robots.txt.

Once you manually exclude the page in your robots.txt doc, the search engines will no longer crawl and index the page. After enough time passes, the page should disappear from the SE's cache.

Here is a moz tutorial for how to exclude the page: Robot's Exclusion Protocol

Just a heads up..you may want to give it a week or so for the SEs to catch up on all the work you have already done to resolve the issue. Then try the above solution.

Good luck!

[DanDeceuster]

You have redirected the index.php version to the / version and it doesn't work? Sounds like you made an error in your .htaccess file then. Make sure your redirects are correct and that every index.php redirects to the / version of the URL and then use the canonical tag to specify the / version as the one you want. Wait a couple weeks and it should get fixed just fine. If it isn't, you probably didn't set up the 301 redirects properly.