Layered navigation and hiding nav from user agent
-
I am trying to deal with the duplicate content issues presented by Magento's layered navigation feature (aka faceted navigation). I installed Amasty's Improved Navigation extension (https://amasty.com/improved-layered-navigation.html) and it offers the option to hide the layered navigation from specific user agents (ie googlebot, bingbot, etc).
This seems like cloaking to me and I hesitate to try it, unless hiding faceted navigation from specific user agents is known to be acceptable to Google (white hat practice). Does anyone know if this the case?
-
Great, thanks Carson! You're insights have been very helpful. I think we'll try to make the on-page ajax solution work.
-
If you're really worried about indexation I think that's a fine solution. It's definitely easier to manage, and it'll also be easier to track pageviews in most analytics platforms. The only downside is that if someone emails or links to a category page with filters applied the recipient won't see it. But generally people share products and not category pages, so it's not a big deal. I'd probably go that route.
Also make sure that your category pages still update the URL when you go to page 2, or that page 2 is somehow also being indexed. You don't want products that don't get indexed because categories can't be crawled.
-
Thanks for the link! I can see how Google offers me a way to tell it how to use my site variables. It seems like between managing parameters in webmaster tools, using canonical links and adding meta noindex tags on variable pages, there should be enough to keep things in order with the search engines. And I can just assume Google knows not to waste too much crawl budget on the variable pages.
I was considering one other option that would remove concerns about variables altogether. Using a different extension, I can set up Magento's layered navigation to work on the page without updating the URL. This eliminates the need for canonicals, parameters, and everything else that is more in Google's control than mine. What do you think of that as a solution?
-
Yes, the bots will crawl the pages, but they will not INDEX them.
There is a concern there, but mostly if the bots get caught in some kind of crawl trap - where they're trying out a near-infinite set of variables and getting stuck in a loop. Otherwise the spiders should understand the variables. You can actually check it in Webmaster tools to make sure Google understands. Instructions for that here:
https://support.google.com/webmasters/answer/6080550?hl=en
Ultimately Google will definitely not penalize you for having lots of duplicate content on URLs through variables, but it might be an issue with Googlebot not finding all your pages. You can make sure that doesn't happen by checking the indexation of your sitemap.
You could also try to block any URLs with the URL parameter in robots.txt. Make sure you get some help on the RegEx if you plan to do this. My advice is that blocking the variables in robots.txt is not worth it, as Google should have no problems with the variables - especially if the canonical tags are working.
Googlebot at least is smart enough these days to know when to stop crawling variable pages, so I think there are more important on-site things to worry about. Make sure your categories are linked to and optimized, for example.
-
This gets into an issue of bots and crawling where I am less clear. Even with canonicals, don't search engine bots crawl all of the pages produced with faceted navigation? That will easily reach 10,000+ pages on my site, which currently has a total number of pages in the low hundreds. I was under the impression I don't want to set up the faceted navigation in a way where the bots crawl through every combination of pages created by my products' attribute filters and bog the bots down in a quagmire of low-value pages. But I'm not sure if that's the case or how concerned I need to be about the bots spending their time on those pages.
-
If I'm not mistaken Magento has canonical tags on category pages by default, so you might be trying to solve an issue that doesn't exist. Take a look at the source code on faceted navigation to confirm. Or you can send me the site and I'll look over it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Will google be able to crawl all of the pages given that the pages displayed or the info on a page varies according to the city of a user?
So the website I am working for asks for a location before displaying the product pages. There are two cities with multiple warehouses. Based on the users' location, the product pages available in the warehouse serving only in that area are shown. If the user skips location, default warehouse-related product pages are shown. The APIs are all location-based.
Intermediate & Advanced SEO | | Airlift0 -
How to manage user profiles in your website?
We have a real estate website in which agents and builders can create their profiles. My question is shall we use h1 or h2 tags in business profile pages or make them according to web 2.0 standards? In case header tags are used, if two agents have the same name and we have used h2 tag for them, then search result page will end up having two same h2's. Can someone please tell me the right way to manage business profiles in a website? Thanks
Intermediate & Advanced SEO | | dailynaukri1 -
SEO page descriptions on mobile - how to hide while preserving the juice for SEO?
Hi everybody, On our pages we have crafted good text paragraphs for SEO purposes. On desktop everything is fine but on mobile the paragraph of text pushes the main content really low on the page. Is there a way of hiding the text while preserving the SEO juices and not getting penalised by Google for spamming techniques? I'd appreciate any recommendations on how to deal with this. Thanks very much!
Intermediate & Advanced SEO | | Firebox0 -
Can using nofollow on magento layered navigation hurt?
Howdy Mozzers! We would like to use no follow, no index on our magento layered navigation pages after any two filters are selected. (We are using single filter pages as landing page, so we would liked them indexed) Is it ok to use nofollow, noindex on these filter pages? Are there disadvantages of using nofollow on internal pages? Matt mentioned refraining from using nofollow internally https://www.youtube.com/watch?v=4SAPUx4Beh8 But we would like to conserve crawling bandwidth and PR flow on potentially 100's of thousands of irrelevant/duplicate filter pages.
Intermediate & Advanced SEO | | MozAddict0 -
Spellcheck necessary for user generated content?
We have a lot of user generated reviews on our key landing pages. Matt Cutts recommended using correctly spelled content. Would you perform a spellcheck of all already published user reviews or would you leave already published reviews rather intact and only perform spellcheck for new reviews before they are published? Since reviews have been marked up using schema.org, I am not sure whether posterior editing of lots of reviews may raise a flag with google regarding manipulating reviews. Thanks.
Intermediate & Advanced SEO | | lcourse0 -
How to solve a PHP problem to increase user experience ?
Hi all, I am having a peculiar PHP problem that is affecting the user experience of the site and thus, the SEO. The site is www.atlantiksurf.com and as you can see I use the QTranslate Plugin to manage the 3 different languages: www.atlantiksurf.com (Spanish) www.atlantiksurf.com/en www.atlantiksurf.com/de If you enter the german version: http://www.atlantiksurf.com/de/ , and you scroll down to the bottom of the page, you will notice there are a couple of posts that have (Español) in their Titles. This means that this particular post is written in Spanish and should only appear in the Spanish version of the site. Every single post is published on the three versions even when they are language orientated. The result is that when you press that specific Spanish title on the German version, you get nothing but this: http://www.atlantiksurf.com/de/aritz-aranburu-tendra-que-estar-un-mes-sin-competir/ Because obviously there is nothing there. If I go to the wordpress admin panel and search for the post manually, I can solve the problem by erasing all the default code of the different language versions that should not appear. But this is only a manual and non practical solution. The problem, I think, might be in the PHP orders that the Wordpress Theme is receiving from somewhere. I come to all of you SEomoz users as my last chance, because I've been months in discussions with Qtranslate users and php amateurs that cannot solve the issue. I know that this question isn't strictly about SEO, but in a way it is, because it must be affecting the way that Google look at us. Please, some help or orientation would be highly appreciated. rTqZI.png
Intermediate & Advanced SEO | | Tintanus0 -
Local Search without the user typing local?
Hi, I'm a somewhat regionally based voip provider for businesses. So I'm not interested in getting the #1 ranking for voip, but I'd like to get the top for my region. So in this case asheville voip and related searches. However, I know that alot of users in Asheville are not typing in Asheville voip when they google. They're just typing in voip or free voip, or cisco voip. Here's my Google Insight Search: http://www.google.com/insights/search/#q=voip&geo=US-NC&date=today%2012-m&cmpt=q So what I was thinking about doing was in addition to my main site. Building several smaller 'educational based sites' about the benefits of VOIP. Based on google insights something like ashevillevoipphone.com. And use it to capture leads and link to my main site. So my question is this: Is this a good strategy? If people in Asheville are just typing in voip phone, will ashevillevoipphone.com automatically have a better chance at a higher ranking? Thanksd David
Intermediate & Advanced SEO | | StraightRazorDesigns0 -
Best SEO Practices for Top-Level Navigation Structure
OK - First of all, thank you to those of you who view and take the time to answer our question. We are currently in the middle of re-designing our golf packages website, and we're trying to decide the best way to structure our Main Navigation for maximum SEO benefit while keeping user experience in mind. The top key phrases we are currently targeting: 1) Myrtle Beach Golf 2) Myrtle Beach Golf Packages You can find the current navigation structure we have come up with here: http://www.myrtlebeachsitemasters.com/index2.html So our question is this: We have subdivisions of: Golf Packages, Accommodations, Golf Courses Is it in our best interest to: A) Get rid of the subdivisions and consolidate them to one page? or B) Simply "NoFollow" the subdivisions within the Main Navigation? We are concerned about the subdivisons for 2 reasons: Too many internal links in Main Navigation The "first link only" rule with Google affecting our additional internal links on existing pages. THANK YOU again to those of you who take the time to answer this question. We really appreciate any clarification on this issue.
Intermediate & Advanced SEO | | JamesO0