Layered navigation and hiding nav from user agent
-
I am trying to deal with the duplicate content issues presented by Magento's layered navigation feature (aka faceted navigation). I installed Amasty's Improved Navigation extension (https://amasty.com/improved-layered-navigation.html) and it offers the option to hide the layered navigation from specific user agents (ie googlebot, bingbot, etc).
This seems like cloaking to me and I hesitate to try it, unless hiding faceted navigation from specific user agents is known to be acceptable to Google (white hat practice). Does anyone know if this the case?
-
Great, thanks Carson! You're insights have been very helpful. I think we'll try to make the on-page ajax solution work.
-
If you're really worried about indexation I think that's a fine solution. It's definitely easier to manage, and it'll also be easier to track pageviews in most analytics platforms. The only downside is that if someone emails or links to a category page with filters applied the recipient won't see it. But generally people share products and not category pages, so it's not a big deal. I'd probably go that route.
Also make sure that your category pages still update the URL when you go to page 2, or that page 2 is somehow also being indexed. You don't want products that don't get indexed because categories can't be crawled.
-
Thanks for the link! I can see how Google offers me a way to tell it how to use my site variables. It seems like between managing parameters in webmaster tools, using canonical links and adding meta noindex tags on variable pages, there should be enough to keep things in order with the search engines. And I can just assume Google knows not to waste too much crawl budget on the variable pages.
I was considering one other option that would remove concerns about variables altogether. Using a different extension, I can set up Magento's layered navigation to work on the page without updating the URL. This eliminates the need for canonicals, parameters, and everything else that is more in Google's control than mine. What do you think of that as a solution?
-
Yes, the bots will crawl the pages, but they will not INDEX them.
There is a concern there, but mostly if the bots get caught in some kind of crawl trap - where they're trying out a near-infinite set of variables and getting stuck in a loop. Otherwise the spiders should understand the variables. You can actually check it in Webmaster tools to make sure Google understands. Instructions for that here:
https://support.google.com/webmasters/answer/6080550?hl=en
Ultimately Google will definitely not penalize you for having lots of duplicate content on URLs through variables, but it might be an issue with Googlebot not finding all your pages. You can make sure that doesn't happen by checking the indexation of your sitemap.
You could also try to block any URLs with the URL parameter in robots.txt. Make sure you get some help on the RegEx if you plan to do this. My advice is that blocking the variables in robots.txt is not worth it, as Google should have no problems with the variables - especially if the canonical tags are working.
Googlebot at least is smart enough these days to know when to stop crawling variable pages, so I think there are more important on-site things to worry about. Make sure your categories are linked to and optimized, for example.
-
This gets into an issue of bots and crawling where I am less clear. Even with canonicals, don't search engine bots crawl all of the pages produced with faceted navigation? That will easily reach 10,000+ pages on my site, which currently has a total number of pages in the low hundreds. I was under the impression I don't want to set up the faceted navigation in a way where the bots crawl through every combination of pages created by my products' attribute filters and bog the bots down in a quagmire of low-value pages. But I'm not sure if that's the case or how concerned I need to be about the bots spending their time on those pages.
-
If I'm not mistaken Magento has canonical tags on category pages by default, so you might be trying to solve an issue that doesn't exist. Take a look at the source code on faceted navigation to confirm. Or you can send me the site and I'll look over it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate without user-selected canonical excluded
We have pdf files uploaded in the media of wordpress and used in our website. As these pdfs are duplicate content of the original publishers, we have marked links to these pdf urls as nofollow. These pages are also disallowed in robots.txt Now, Google Search Console has shown these pages Excluded as "Duplicate without user-selected canonical" As it comes out we cannot use canonical tag with pdf pages so as to point to the original pdf source If we embed a pdf viewer in our website and fetch the pdfs by passing the urls of the original publisher, would the pdfs be still read as text by google and again create duplicate content issue? Another thing, when the pdf expires and is removed, it would lead to 404 error. If we direct our users to the third party website, then it would add up to our bounce rate. What should be the appropriate way to handle duplicate pdfs? Thanks
Intermediate & Advanced SEO | | dailynaukri1 -
Title Tags in Sitecore are the same as navigation. How do I add keyword phrases without effecting my website's navigation?
I am working on overhauling the on-page SEO for ecommerce website on Sitecore. I've done all my research and I am ready to plug the Title tags and descriptions in. So, if the page on Navigation is 'SHOP' this is in the Title tag box. How do I add my 70 characters of keywords? Thanks. JOE
Intermediate & Advanced SEO | | iJoe0 -
Navigation Menu - Whats too much
Ive always had pages set up for a lot of our products and had these in the navigation menu. For instance we sell Solar Control Window Film which helps with heat, glare and UV. We then have a navigation menu something like this: Solar Window Film
Intermediate & Advanced SEO | | Fozzy1609
Heat Control window Films
Anti glare window film
UV window film
etc etc Ihave this for all my services and products. I have unique content on each. My question is this. Would I be better having the naviation menu with links to all the seperate services we offer
OR
Should I have it linking to the main services and then the related services from within the page> For example Ill have just Solar Window Film in the navigation and then on the page it would internally link to the heat related section and the glare related section etc. Im wondering whether my sub pages would suffer because theyre not linked to from every page with the second method or whether it would help in some way0 -
What is best practise for markup in a nav bar, avoid duplicate content.
I'm wondering what's best practise for markup in the nav bar. So if we have <nav> Can we have headers in there, does it help? If we do, would a H1 confuse google as to wether that is the H1 of importance of the one in thein the </nav>
Intermediate & Advanced SEO | | TrueluxGroup0 -
HTML5 Nav Tag Issue - Be Aware
In checking my internal links with GWT, it is apparent that links within the nav tag in HTML5 are discounted by Google as "internal links" This could have major repercussions for designing your internal link structure for SEO purposes. I was surprised to see this result, as I have never seen it discussed. Anyone else notice this, or have any alternative views?
Intermediate & Advanced SEO | | veezer0 -
How to Hide Directories in Search?
I noticed bad 404 error links in Google Webmaster Tools and they were pointing to directories that do not have an actual page, but hold information. Ex: there are links pointing to our PDF folder which holds all of our pdf documents. If i type in , example.com/pdf/ it brings up a unformated webpage that displays all of our PDF links. How do I prevent this from happening. Right now I am blocking these in my robots.txt file, but if i type them in, they still appear. Or should I not worry about this?
Intermediate & Advanced SEO | | hfranz0 -
Local Search without the user typing local?
Hi, I'm a somewhat regionally based voip provider for businesses. So I'm not interested in getting the #1 ranking for voip, but I'd like to get the top for my region. So in this case asheville voip and related searches. However, I know that alot of users in Asheville are not typing in Asheville voip when they google. They're just typing in voip or free voip, or cisco voip. Here's my Google Insight Search: http://www.google.com/insights/search/#q=voip&geo=US-NC&date=today%2012-m&cmpt=q So what I was thinking about doing was in addition to my main site. Building several smaller 'educational based sites' about the benefits of VOIP. Based on google insights something like ashevillevoipphone.com. And use it to capture leads and link to my main site. So my question is this: Is this a good strategy? If people in Asheville are just typing in voip phone, will ashevillevoipphone.com automatically have a better chance at a higher ranking? Thanksd David
Intermediate & Advanced SEO | | StraightRazorDesigns0 -
What is the Ideal Structure for User Generated Product Reviews on My Site?
I apologize for the lengthy post, but I need help! Here is my current structure for product reviews: My product pages displays a set number of user product reviews before displaying a link to "see all reviews". So: http://www.domain.com/product/product-page Has product details, specs (usually generic from manufacturer) and 5 user product reviews. If there are more than 5, there is a link to see all reviews: http://www.domain.com/reviews/product-page?page=1 Where each page would display 10 user product reviews, and paginate until all user reviews are displayed. I am thinking about using the Rel Canonical tag on the paginated reviews pages to reference back to the main product page. So: http://www.domain.com/reviews/product-page?page=1 http://www.domain.com/reviews/product-page?page=2 http://www.domain.com/reviews/product-page?page=3 Would have the canonical URL of: http://www.domain.com/product/product-page Does this structure make sense? I'm unclear what strategy I should use, but currently the product review pages account for less than 2% of overall organic traffic. Thanks ahead of time!
Intermediate & Advanced SEO | | Corp0