Can using nofollow on magento layered navigation hurt?

[MozAddict]

Howdy Mozzers!

We would like to use no follow, no index on our magento layered navigation pages after any two filters are selected. (We are using single filter pages as landing page, so we would liked them indexed)

Is it ok to use nofollow, noindex on these filter pages? Are there disadvantages of using nofollow on internal pages?

Matt mentioned refraining from using nofollow internally https://www.youtube.com/watch?v=4SAPUx4Beh8

But we would like to conserve crawling bandwidth and PR flow on potentially 100's of thousands of irrelevant/duplicate filter pages.

[AGMContainerControls]

I understand I might be a little late, but I had experienced this issue first hand with a Magento site. Once I added a wildcard exclusion in the robots.txt file my impressions and click improved noticeably.

[LynnPatchett]

HI,

That is quite a few pages!

If the main issue is crawling related then robots.txt is probably the best way to go, I think the meta tags will still allow the pages to be crawled (they have to be for the tag to be read). Check out the comments in this and this post for wildcard matching in robots.txt which should do what you need. If the pages are indexed then it might be wise to leave a bit of time so that the noindex tags are picked up and then implement the crawl blocking in the robots.txt (and test in GWT to make sure you are not accidentally blocking more then you think). In this case I think you could still leave out the nofollow meta tag but this might just be personal opinion now - I'm not sure if in practice it would make much difference once you have no indexed and blocked crawling!

[MozAddict]

Hi Lynn,

Thank you for your valuable input on the matter. Yes, using meta tags in the header. We are currently submitting filter pages that we want indexed through the site map, so google bot should be able to reach these pages. Also, we are displaying noindex, nofollow tags only on filter pages which have a combination of more than two filters selected as we do not need to go any deeper than that.

I understand your point of using noindex, follow instead of noindex, nofollow to prevent unexpected crawl issues. But on the contrary, don't you think we could conserve crawling bandwidth using noindex, nofollow tags on filter pages that serve no purpose being crawled and probably wont be externally linked to either?

We currently have around 7 filters, some with many values. This can create combinations of more than 500,000 filter pages...

Thanks

[LynnPatchett]

Hi,

I assume you mean in a meta header tag for these pages? As a general rule I would avoid using nofollow and simply noindex the pages in question. If you are implementing this with a meta tag then the pages will be reached from the layered navigation links anyway so they would then be a dead end for both PR and the crawler - with the potential to cause unexpected crawl problems rather than optimising it.

As long as you are addressing as best you can any duplicate content issues caused by the layered navigation (check out this post for a good rundown on the various solutions) then I would leave the noindex in place and let the crawler follow the links as normal.