Hi!

We do a lot of consultancy for Magento projects and this is a question that comes up quite regularly as it can't really be handled perfectly straight out of the box with Magento.

Every implementation is a little bit different, but I'll put together some recommendations below based on the information available at the moment.

For your faceted navigation, you ideally don't want to index any of these pages, unless you believe that you'll rank in your own right for specific filters (e.g. Colour, like you pointed out in your last message).

That then comes with some additional complications. In Magento, if you have 3 colours available in the faceted nav, you'll have all the different variations indexed in each combination.

For example:

Blue
Black
Red

Blue + Black
Blue + Red
Black + Red
Black + Blue
Red + Blue
Red + Black

Magento as standard doesn't always keep the filters in the same order, so you can end up with literally thousands of pages ending up in the index for a relatively small number of attributes being shown on your pages.

There are a few recommendations here:

1. Go and look at the MageWorx Ultimate SEO Suite Plugin - http://www.mageworx.com/seo-suite-ultimate-magento-extension.html - For $249, it solves a lot of issues Magneto has straight out of the box and gives you ultimate control over your meta titles.

What you want to do is set all of your facets to 'NOINDEX,FOLLOW' where possible. This will reduce the number of URLs in the index gradually. An example of this would be adding ?min=* and mode=* etc (grid/list variants).

2. For your canonicals, you're probably best setting the canonical to the current filtered page (for example, if you're on a category page with colour = blue selected in your faceted nav, you'd have this URL as your canonical). Some sites we work on have it setup so the canonical points to the category URL (like you currently have).

Finally, you probably want to build an extension to allow you to inject content into the filtered content pages. If you're using an extension like ManaDev for your facet navigation, this can be achieved fairly easily and allows you to add a block of text to each filter applied on a page.

You should also look to request each of the incorrectly indexed URLs is removed from the index (although this does take a long time if you have a lot!).

We wrote a really long guide around launching a Magento website last month which may be of interest - https://www.pinpointdesigns.co.uk/the-definitive-guide-to-launching-a-magento-website/. We've also done a guide on Common Magento SEO Issues here - https://www.pinpointdesigns.co.uk/common-magento-seo-issues/ and I previously wrote a guide on setting Magento up for Search Engines on Moz - https://moz.com/ugc/setting-up-magento-for-the-search-engines (Although this is likely to be a little outdated now)

I hope this helps!
Lewis

You're right in with the **Disallow: /fish **in the robots file blocking all those initial links, but if you wanted to block everything inside the /en/ folder, you would need to do disallow: /en/fish

You could use a wildcard in the robots.txt file to do something along the lines of Disallow: /BTS-*

This _'should' _work, but it's always worth checking using a tool to make sure it's all implemented correctly. Distilled did a post a while back about a JS tool which allows you to test if robots.txt files work correctly which can be found here - http://www.distilled.net/blog/seo/js-bookmarklet-for-checking-if-a-page-is-blocked-by-robots-txt/

In addition to this, you could also use the 'blocked URLs' tool in GWT to see if the pages are successfully blocked once you've implemented the code.

Hope this helps!