Layered navigation and hiding nav from user agent
-
I am trying to deal with the duplicate content issues presented by Magento's layered navigation feature (aka faceted navigation). I installed Amasty's Improved Navigation extension (https://amasty.com/improved-layered-navigation.html) and it offers the option to hide the layered navigation from specific user agents (ie googlebot, bingbot, etc).
This seems like cloaking to me and I hesitate to try it, unless hiding faceted navigation from specific user agents is known to be acceptable to Google (white hat practice). Does anyone know if this the case?
-
Great, thanks Carson! You're insights have been very helpful. I think we'll try to make the on-page ajax solution work.
-
If you're really worried about indexation I think that's a fine solution. It's definitely easier to manage, and it'll also be easier to track pageviews in most analytics platforms. The only downside is that if someone emails or links to a category page with filters applied the recipient won't see it. But generally people share products and not category pages, so it's not a big deal. I'd probably go that route.
Also make sure that your category pages still update the URL when you go to page 2, or that page 2 is somehow also being indexed. You don't want products that don't get indexed because categories can't be crawled.
-
Thanks for the link! I can see how Google offers me a way to tell it how to use my site variables. It seems like between managing parameters in webmaster tools, using canonical links and adding meta noindex tags on variable pages, there should be enough to keep things in order with the search engines. And I can just assume Google knows not to waste too much crawl budget on the variable pages.
I was considering one other option that would remove concerns about variables altogether. Using a different extension, I can set up Magento's layered navigation to work on the page without updating the URL. This eliminates the need for canonicals, parameters, and everything else that is more in Google's control than mine. What do you think of that as a solution?
-
Yes, the bots will crawl the pages, but they will not INDEX them.
There is a concern there, but mostly if the bots get caught in some kind of crawl trap - where they're trying out a near-infinite set of variables and getting stuck in a loop. Otherwise the spiders should understand the variables. You can actually check it in Webmaster tools to make sure Google understands. Instructions for that here:
https://support.google.com/webmasters/answer/6080550?hl=en
Ultimately Google will definitely not penalize you for having lots of duplicate content on URLs through variables, but it might be an issue with Googlebot not finding all your pages. You can make sure that doesn't happen by checking the indexation of your sitemap.
You could also try to block any URLs with the URL parameter in robots.txt. Make sure you get some help on the RegEx if you plan to do this. My advice is that blocking the variables in robots.txt is not worth it, as Google should have no problems with the variables - especially if the canonical tags are working.
Googlebot at least is smart enough these days to know when to stop crawling variable pages, so I think there are more important on-site things to worry about. Make sure your categories are linked to and optimized, for example.
-
This gets into an issue of bots and crawling where I am less clear. Even with canonicals, don't search engine bots crawl all of the pages produced with faceted navigation? That will easily reach 10,000+ pages on my site, which currently has a total number of pages in the low hundreds. I was under the impression I don't want to set up the faceted navigation in a way where the bots crawl through every combination of pages created by my products' attribute filters and bog the bots down in a quagmire of low-value pages. But I'm not sure if that's the case or how concerned I need to be about the bots spending their time on those pages.
-
If I'm not mistaken Magento has canonical tags on category pages by default, so you might be trying to solve an issue that doesn't exist. Take a look at the source code on faceted navigation to confirm. Or you can send me the site and I'll look over it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Will exit intent layer harm google ranking? What if third party ad in exit intent layer?
We plan to implement an exit intent layer.
Intermediate & Advanced SEO | | lcourse
For mailing list expansion and/or to display personalized third party advertising. Any negative impact this may have on google rankings?
Shall we limit exit intent layer so to show only to users who visited at least 2 pages on our site?
Shall we reduce size of layer on mobile?
Anything else to consider to minimize potential negative impact on google rankings?0 -
How to find Topics ? software or results and user intent
Hello, Is there a software that is better than an other to find to right topics to cover in my content. I am thinking about Moz, Marketmuse or Semrush or is it better to look at the search results because they match user intent and see what is covered and cover those in my content Thank you,
Intermediate & Advanced SEO | | seoanalytics1 -
Duplicate content issue with pages that have navigation
We have a large consumer website with several sections that have navigation of several pages. How would I prevent the pages from getting duplicate content errors and how best would I handle SEO for these? For example we have about 500 events with 20 events showing on each page. What is the best way to prevent all the subsequent navigation pages from getting a duplicate content and duplicate title error?
Intermediate & Advanced SEO | | roundbrix0 -
Jump to Navigation in SERPs?
To make 'jump to' navigation work, does the href or anchor need to contain descriptive text? For example, I know this is best: Install with Wubi But, would the below work just as well? Install with Wubi
Intermediate & Advanced SEO | | nicole.healthline0 -
Do total number of links in main navigation play a role in ranking?
On one site I work on, I noticed that pages in the upper navigation that have less links in the main navigation vs pages that have more links tend to perform better in organic search. Has anyone else noticed this?
Intermediate & Advanced SEO | | nicole.healthline0 -
How do I reduce internal links & cannibalisation from primiary navigation?
SEOmoz tools is reporting each page on our site containing in excess of 200 internal links mostly from our primary navigation menu which it says is too many. This also causes cannibalization on the word towels which i would like to avoid if possible. Is there a way to reduce the number of internal links whilst maintaining a good structure to allow link juice to filter through the site and also reduce cannibalization?
Intermediate & Advanced SEO | | Towelsrus0 -
How to Hide Directories in Search?
I noticed bad 404 error links in Google Webmaster Tools and they were pointing to directories that do not have an actual page, but hold information. Ex: there are links pointing to our PDF folder which holds all of our pdf documents. If i type in , example.com/pdf/ it brings up a unformated webpage that displays all of our PDF links. How do I prevent this from happening. Right now I am blocking these in my robots.txt file, but if i type them in, they still appear. Or should I not worry about this?
Intermediate & Advanced SEO | | hfranz0 -
Best SEO Practices for Top-Level Navigation Structure
OK - First of all, thank you to those of you who view and take the time to answer our question. We are currently in the middle of re-designing our golf packages website, and we're trying to decide the best way to structure our Main Navigation for maximum SEO benefit while keeping user experience in mind. The top key phrases we are currently targeting: 1) Myrtle Beach Golf 2) Myrtle Beach Golf Packages You can find the current navigation structure we have come up with here: http://www.myrtlebeachsitemasters.com/index2.html So our question is this: We have subdivisions of: Golf Packages, Accommodations, Golf Courses Is it in our best interest to: A) Get rid of the subdivisions and consolidate them to one page? or B) Simply "NoFollow" the subdivisions within the Main Navigation? We are concerned about the subdivisons for 2 reasons: Too many internal links in Main Navigation The "first link only" rule with Google affecting our additional internal links on existing pages. THANK YOU again to those of you who take the time to answer this question. We really appreciate any clarification on this issue.
Intermediate & Advanced SEO | | JamesO0