Best method to stop crawler access to extra Nav Menu
-
Our shop site has a 3 tier drop down mega-menu so it's easy to find your way to anything from anywhere. It contains about 150 links and probably 300 words of text.
We also have a more context-driven single layer of sub-category navigation as well as breadcrumbs on our category pages.
You can get to every product and category page without using the drop down mega-menu.
Although the mega-menu is a helpful tool for customers, it means that every single page in our shop has an extra 150 links on it that go to stuff that isn't necessarily related or relevant to the page content. This means that when viewed from the context of a crawler, rather than a nice tree like crawling structure, we've got more of an unstructured mesh where everything is linked to everything else.
I'd like to hide the mega-menu links from being picked up by a crawler, but what's the best way to do this?
I can add a nofollow to all mega-menu links, but are the links still registered as page content even if they're not followed? It's a lot of text if nothing else.
Another possibility we're considering is to set the mega-menu to only populate with links when it's main button is hovered over. So it's not part of the initial page load content at all.
Or we could use a crude yet effective system we have used for some other menus we have of base encoding the content inline so it's not readable by a spider.
What would you do and why?
Thanks,
James
-
I agree Alan,
Mega Menu's are a good way to dilate the link equity of your page and in most cases it isn't needed at all. Keep the top-level navigation simple and have a submenu on all pages that contain links relevant to that section.
EG: Mega Menu could be:
Home Mens (Mens Tops, Mens Jeans, Mens Coats), Women (Womens Tops, Womens Jeans etc) Contact us
In this example it would be better to have one top level menu for:
Home | Mens | Women | Contact us
Then when your in the men or women section show links to "Tops", "Jeans" and "Coats". That way those links are relevant to the section you're in and reinforces the structure of that section to search engines.
After giving it further thought I would suggest not having a mega menu at all, because it may harm your SEO on-page optimisation efforts in the long term.
-
Ben's partially correct. Unfortunately Google has been claiming they do process Javascript for a while, and they recently stated they've begun reading AJAX. Of course they do a lousy job of it and don't always get it right, which just makes things even more muddy.
So from an SEO best practices perspective, you shouldn't have the menu(s) in the first place, at all.
You may also THINK their good for users but has any significant study been performed to confirm that? You'd need to check click-through rates on all the links to know for sure.
What I've found through years of auditing sites that have such menus is that it almost always turns out to be the case where most of the deeper links NEVER get clicked on from within these menus. Instead, they're overwhelming to users. This is why it's better to not have them from a UX perspective.
If you abandon them and go with more traditional hierarchical index and sub-index pages, and if those are properly optimized, you'll not only eliminate the massive SEO problem but in fact get more of your category pages to have higher ranking strength and authority over time.
IF you're going to keep them in any form because you don't want to go to the extreme I recommend, then yes - AJAX would likely be the only scenario that offers the least likelihood of search engines choking on the over-use of links.
And for the record, the real current problem with all those links on every page is duplicate content confusion - all of those URLS at the source level dilutes the uniqueness of content on every page of the site. And that also means you're harming the topical focus of every page as well. So whatever you do, AJAX or doing away with them altogether is going to be of high value long term.
- Alan Bleiweiss - Click2Rank's Search Team Director
-
From my experience I don't think you can really 'hide' the megamenu links from a crawler if they are generated using a content management system (code server side). If the link is on the page in the HTML then it will be crawled by a bot etc.
The general method of getting a mega menu to work is through the use of CSS and JavaScript, so you might want to have a look at using AJAX to get the relevant links from the database and then use JavaScript to put the links into the page.
This isn't a great solution, but bots cannot load JavaScript, so what they will see is only the links that are served up from the content management system.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What is the best way to employ log-in to benefit in SEO?
Hi all, All SaaS companies have this log-in page as their top visited page in their websites and some times it helps and also hurts them. I've gone through some big SaaS companies websites and they handle the log-in page differently like on sub domain, on website page, some will directly link to their instance login without a page, etc...I wonder what is the best practice to host the log-in to make sure the more visits to log-in page don't hurt us but give us some boost. Thanks
Web Design | | vtmoz0 -
What's the world's best hosting?
Hello folks, I'm looking at hosting options. In your opinion, what's the best provider out there and why? Cheers, Gill.
Web Design | | Cannetastic0 -
H1 tag within on top nav but css class styled appear at bottom
Hi, Sorry the similar question could have been ask previously but I couldn't get an exact answer. Someone put a h1 tag on our homepage (which do not have before) and the is within the top nav. But it has a css style class in that h1 tag to make it appear at the bottom of the page so not everyone would be aware of that small h1 title. I personally do not think that's going to help with seo. However I want to know if this practice is going to be: beneficial, or not at all? or harmful instead? Thanks LM
Web Design | | LauraHT0 -
Best strategy for multiple internal links
We have a website that has a whole heap of internal links. After re-structuring the footer links to the main categories we are now looking at the best way to display the internal links from each category (approx 20-50 links from each). At present they are simply listed on the corresponding category page, however this doesnt look great so we want to display them in a way that is user friendly and also seo friendly. Any suggestions and examples appreciated. Thanks in advance.
Web Design | | Silkstream0 -
What pluggin translator is the newest and the best to install on wordpress. Thank you very much
What pluggin translator is the newest and the best to install on wordpress. Thank you very much
Web Design | | maestrosonrisas0 -
How can we improve our e-commerce site architecture to help best preserve Page Authority?
Today I installed the SEOMoz toolbar for Firefox (very cool, highly recommended). I was comparing our site http://www.ccisolutions.com to this competitor: http://www.uniquesquared.com For the most part, the deeper I go in our site the more the page authority drops. We have a few exceptions where the page authority of a subcategory page is actually better than the cat. page one level up. In comparison, when I was looking at http://www.uniquesquared.com I noticed that their page authority stays at "21" on every single category page I visit. Are you seeing what I'm seeing? Is this potentially a problem with the tool bar or, is there something significantly different about their site architecture that allows them to maintain that PA across all category and sub category pages? Is there something fundamentally wrong with our (http://www.ccisolutions.com) site architecture? I understand that we have longer URLs, but this is an old store with a lot of SKUs, so we have decided not to remove the /category/ and /product/ from the URLs because the 301 redirects that would result wouldn't pass all of the authority they've built up over the years. Interested to know viewpoints on the site architecture and how it might be improved. Thanks!
Web Design | | danatanseo0 -
Best hosting solution for linkbait videos
I'm creating a series of link bait videos. I want people to link to the pages on my site hosting the videos. What's the best site to host them on, if my primary concern is to get do-followed links? I'm worried about putting them on YouTube, as I imagine most of the links will be to the page on youtube.com, or worse, just directly embedded.
Web Design | | menachemp0 -
Best E-commerce Solution - SEO Friendly
I need to know thoughts on the best ecommerce solution for our company. We currently have one website with our products, that people call and they purchase over the phone. In the future we are considering adding a different product line, to which we want to be a fully functional online ecommerce site. We eventually are considering having only a portion of our products on our current site to have the ability to purchase online. The reasoning is because we sell very high quality products that range from $1000 - $200,000. We would like to have a consumer section and an industrial sectioin. The consumer section would consist of products from $1000 - $10,000 or so. Then the industrial products you would need to call to purchase. So would it make sense to have a main website that is our corporate site with links to different websites? Or to keep this all in one website and have different directories like so: corporatecompany.com - links to product-line.com corporatecompany.com - links to product-line-two.com or corporatecompany.com/consumer-grade/ corporatecompany.com/industrial-grade/ corporatecompany.com/the-ecommerce-only-section/ Can a shopping cart be used to turn on half of the products for purchasing online, and the other half to be call in only ? What is the best e-commerce solution that is SEO friendly and also can just play a role of a regular website?
Web Design | | hfranz0