Layered navigation and hiding nav from user agent
-
I am trying to deal with the duplicate content issues presented by Magento's layered navigation feature (aka faceted navigation). I installed Amasty's Improved Navigation extension (https://amasty.com/improved-layered-navigation.html) and it offers the option to hide the layered navigation from specific user agents (ie googlebot, bingbot, etc).
This seems like cloaking to me and I hesitate to try it, unless hiding faceted navigation from specific user agents is known to be acceptable to Google (white hat practice). Does anyone know if this the case?
-
Great, thanks Carson! You're insights have been very helpful. I think we'll try to make the on-page ajax solution work.
-
If you're really worried about indexation I think that's a fine solution. It's definitely easier to manage, and it'll also be easier to track pageviews in most analytics platforms. The only downside is that if someone emails or links to a category page with filters applied the recipient won't see it. But generally people share products and not category pages, so it's not a big deal. I'd probably go that route.
Also make sure that your category pages still update the URL when you go to page 2, or that page 2 is somehow also being indexed. You don't want products that don't get indexed because categories can't be crawled.
-
Thanks for the link! I can see how Google offers me a way to tell it how to use my site variables. It seems like between managing parameters in webmaster tools, using canonical links and adding meta noindex tags on variable pages, there should be enough to keep things in order with the search engines. And I can just assume Google knows not to waste too much crawl budget on the variable pages.
I was considering one other option that would remove concerns about variables altogether. Using a different extension, I can set up Magento's layered navigation to work on the page without updating the URL. This eliminates the need for canonicals, parameters, and everything else that is more in Google's control than mine. What do you think of that as a solution?
-
Yes, the bots will crawl the pages, but they will not INDEX them.
There is a concern there, but mostly if the bots get caught in some kind of crawl trap - where they're trying out a near-infinite set of variables and getting stuck in a loop. Otherwise the spiders should understand the variables. You can actually check it in Webmaster tools to make sure Google understands. Instructions for that here:
https://support.google.com/webmasters/answer/6080550?hl=en
Ultimately Google will definitely not penalize you for having lots of duplicate content on URLs through variables, but it might be an issue with Googlebot not finding all your pages. You can make sure that doesn't happen by checking the indexation of your sitemap.
You could also try to block any URLs with the URL parameter in robots.txt. Make sure you get some help on the RegEx if you plan to do this. My advice is that blocking the variables in robots.txt is not worth it, as Google should have no problems with the variables - especially if the canonical tags are working.
Googlebot at least is smart enough these days to know when to stop crawling variable pages, so I think there are more important on-site things to worry about. Make sure your categories are linked to and optimized, for example.
-
This gets into an issue of bots and crawling where I am less clear. Even with canonicals, don't search engine bots crawl all of the pages produced with faceted navigation? That will easily reach 10,000+ pages on my site, which currently has a total number of pages in the low hundreds. I was under the impression I don't want to set up the faceted navigation in a way where the bots crawl through every combination of pages created by my products' attribute filters and bog the bots down in a quagmire of low-value pages. But I'm not sure if that's the case or how concerned I need to be about the bots spending their time on those pages.
-
If I'm not mistaken Magento has canonical tags on category pages by default, so you might be trying to solve an issue that doesn't exist. Take a look at the source code on faceted navigation to confirm. Or you can send me the site and I'll look over it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawler doesn't discover the links in the main nav
Hi Moz Community, We have a headless ecom (Magento) client that I'm trying to crawl the site. During the crawl, the tool (Screaming Frog) cannot discover the sub-category URLs in the main navigation when I start crawling via homepage. Similarly, when I start crawling with one of the sub-category page, it doesn't crawl any of the product URLs on the sub-category page itself. When I inspect product and sub-cat URLs through Search Console, they seem as indexed and if I view how Googlebot rendered the sub-category page, I can see the product URLs on the sub-cat page too. If you have any idea what's the issue with Screaming Frog and would like to help me out, I'd be so grateful! Thanks in advance
Intermediate & Advanced SEO | | bbop330 -
User intent
Hello, For the keyword Normandy cycling, it seems according to the result that people are looking for the bike routes. My question : can i rank indicating my favorite bike routes (personal routes) or doIi need to stick to what is already considered as the best biking routes in Normandy, the tour de Manche, the veloscenic, the velo Francette and so forth ? Thank you,
Intermediate & Advanced SEO | | seoanalytics1 -
Hide and display iframes on different devices
I have an iframe on my website, I'd like to hide it when a user is browsing with a mobile device and display a different one for that user (which will be hidden on desktop). Is it possible that Google views it as cloaking? does it qualify as hidden content?
Intermediate & Advanced SEO | | OrendaLtd0 -
H2 vs. H3 Tags for Category Navigation
Hey, all. I have client that uses tags in the navigation for its blog. For example, tags might appear around "Library," "Recent Posts," etc. This is handled through their WordPress theme. This seems fairly standard, but I wonder whether tags are semantically appropriate. Since each blog post is fairly lengthy (about 500-1000 words) with multiple tags, would it be more appropriate to use tags for this menu navigation? Are we cutting into the effectiveness of our tags by using them for menu navigation? The navigation is certainly an important page element, and it structures content, so it seems that it should use some header tag. Anyways, your thoughts are greatly appreciated. I'm a content creator, not an SEO, so this is a bit out of my skillset.
Intermediate & Advanced SEO | | Ask44435230 -
Using rel="nofollow" when link has an exact match anchor but the link does add value for the user
Hi all, I am wondering what peoples thoughts are on using rel="nofollow" for a link on a page like this http://askgramps.org/9203/a-bushel-of-wheat-great-value-than-bushel-of-goldThe anchor text is "Brigham Young" and the page it's pointing to's title is Brigham Young and it goes into more detail on who he is. So it is exact match. And as we know if this page has too much exact match anchor text it is likely to be considered "over-optimized". I guess one of my questions is how much is too much exact match or partial match anchor text? I have heard ratios tossed around like for every 10 links; 7 of them should not be targeted at all while 3 out of the 10 would be okay. I know it's all about being natural and creating value but using exact match or partial match anchors can definitely create value as they are almost always highly relevant. One reason that prompted my question is I have heard that this is something Penguin 3.0 is really going look at.On the example URL I gave I want to keep that particular link as is because I think it does add value to the user experience but then I used rel="nofollow" so it doesn't pass PageRank. Anyone see a problem with doing this and/or have a different idea? An important detail is that both sites are owned by the same organization. Thanks
Intermediate & Advanced SEO | | ThridHour0 -
Responsive design (Showing diffrent pages(icons) for Mobile/Tablet users)
I'm writing this question just to insure that we are implementing the responsive design correctly.Example of pages: http://www.yamsafer.me/en/united-arab-emirates/abu-dhabi/hotel/beach-rotana-abu-dhabiAnother : http://www.yamsafer.me/enCan we show different pages(Enhanced for mobile users) to mobile/Tablets visitors (sure same content) but with new icons that enhance the User experience for mobile/tablet users , while hiding these items to PC, laptop users?.Thanks
Intermediate & Advanced SEO | | Yamsafer.com0 -
Ever Wise to Intentionally Use Javascript for Global Navigation?
I may be going against the grain here, but I'm going to throw this out there and I'm interested in hearing your feedback... We are a fairly large online retailer (50k+ SKUs) where all of our category and subcategory pages show well over 100 links (just the refinement links on the left can quickly add up to 50+). What's worse is when you hover on our global navigation, you see the hover menu (bot sees them as ) of over 80 links. Now I realize the good rule of thumb is not to exceed 100 links on a page (and if you did your math, you can see we already exceeded that well before we let the bots get to the good stuff we really wanted them to crawl in the first place). So... Is it wise to intentionally shield these global nav links from the bots by using javascript?
Intermediate & Advanced SEO | | mrwestern0 -
How to Hide Directories in Search?
I noticed bad 404 error links in Google Webmaster Tools and they were pointing to directories that do not have an actual page, but hold information. Ex: there are links pointing to our PDF folder which holds all of our pdf documents. If i type in , example.com/pdf/ it brings up a unformated webpage that displays all of our PDF links. How do I prevent this from happening. Right now I am blocking these in my robots.txt file, but if i type them in, they still appear. Or should I not worry about this?
Intermediate & Advanced SEO | | hfranz0