Navigation for search
-
We are getting ready to launch a site that has great navigation for users, but it is not so great for search engines. As long as we are ethical about it, does anyone see a downside to detecting a bot user agent and displaying different nav to it? I suppose some could consider it cloaking, but I noticed amazon uses this strategy and they don't seem to be getting a big penalty lol. We are not going to do anything shady with it, just offer the bot a different way to access our content. Any thoughts?
-
I would never recommend altering your site in any way specifically for a bot. Google has repeatedly stated that would be seen as cloaking and you definitely run the risk of a penalty. It can happen any time. Do you really want that cloud over your head?
Even if Amazon is able to do it, you are not Amazon. Despite any statement to the contrary, a huge site like Amazon can do things that you or I may not be able to do. We can debate the fairness of it, but we are not on a level playing field with them.
You seem to really want to do this, so feel free. But x months from now you will likely make a post "My rankings dropped and I can't figure out why", and then someone will notice the setting and that will be the answer.
-
Could you please share the site in question? we might be able to give a much more insightful thought
-
Thanks for your suggestion Alex!
-
All the content will be accessible through the sitemap which will be displayed in both versions (the user and the bot) version of the site. The only difference is the navigation will change from one version to another. It could be perceived as a gray area, even though we really are just making it more friendly to bots not trying to be deceptive at all.
-
And that should be okay if they see the content is the same between your normal version and the spider version, just a difference in presentation. It becomes a problem when you start serving drastically different content.
-
How about adding an indexable version of the navigation and then hiding it with CSS? The crawlers would see that. I haven't thought about whether that'd be a good idea, just a quick thought.
-
We usually operate under the principal that would this be ok if it were manually reviewed? In our plan, I am pretty confident that it would pass. We are planning on making what is displayed to bots very basic text link navigation. Our XML and html sitemaps will correspond well with the bot navigation (so to show G we really are not trying to hide anything). And yes we could make the nav more search friendly, but one of our main goals is to have the site look as clean as possible. As far as I can tell Amazon uses this method to avoid duplicate content issues and make their site generally more crawlable which is why I think it is on the table as an option.
-
Depending on how much you're changing, I think it could be considered cloaking and end up getting you in trouble. You'll get looked at any time you're sniffing for bots and serving up different content. Are there other ways you could make the navigation index-able?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why home Page is not showing showing in search results?
The USA Glitz I search the exact title & keyword but still its not showing though there is no related websites in this term. What can i do ? What kind of strategy needs to follow ? Kindly help me out . Thanks 😢
On-Page Optimization | | henrichjrr4201 -
Google Search Console issue: "This is how Googlebot saw the page" showing part of page being covered up
Hi everyone! Kind of a weird question here but I'll ask and see if anyone else has seen this: In Google Search Console when I do a fetch and render request for a specific site, the fetch and blocked resources all look A-OK. However, in the render, there's a large grey box (background of navigation) that covers up a significant amount of what is on the page. Attaching a screenshot. You can see the text start peeking out below (had to trim for confidentiality reasons). But behind that block of grey IS text. And text that apparently in the fetch part Googlebot does see and can crawl. My question: is this an issue? Should I be concerned about this visual look? Or no? Never have experienced an issue like that. I will say - trying to make a play at a featured snippet and can't seem to have Google display this page's information, despite it being the first result and the query showing a featured snippet of a result #4. I know that it isn't guaranteed for the #1 result but wonder if this has anything to do with why it isn't showing one. VmIqgFB.png
On-Page Optimization | | ChristianMKG0 -
Does PLA hurt my organic google search.
I have experienced a big drop in my organic trafic to my site. But an increase in my cpc...? Does google punish my site for PLA to get higher revenue?
On-Page Optimization | | Egmont0 -
Ecommerce internal search results pages
I'm working on a ecommerce site that allows product search results pages to be sorted a variety of ways (best selling, newest, by price). Each of these search filters creates a new url i.e. /all/best/1 and /all/best/2; /all/new/1 and /all/new/2; etc. These search results pages have been indexed and the site is receiving enough organic traffic from these pages that I don't want to add noindex,follow to them. I am planning on implementing rel=prev,rel=next for each filter, but I'm concerned about duplicate content considering I can't create unique meta data for each page. Should I canonical all pages to the first search results page without filters applied? Or any other ideas on how I should proceed?
On-Page Optimization | | ang0 -
Search Pages outranking Product Pages
A lot of the results seen in the search engines for our site are pages from our search results on our site, i.e. Widgets | Search Results This has happened over time and wasn't intentional, but in many cases we see our search results pages appearing over our actual product pages in search, which isn't ideal. Simply blocking indexing of these pages via robots wouldn't be ideal, at least all at once as we would have that period of time where those Search Results pages would be offline and our product pages would still be at the back of ranking. Any ideas on a strategy to replace these Search Results with the actual products in a way that won't hurt us too bad during the transition? Or a way to make the actual product pages rank above the search results? Currently, it is often the opposite. Thanks! Craig
On-Page Optimization | | TheCraig0 -
Keywords in Navigation
Hi, What is best practice for main navigation links with regards to use of keywords in them. For example is it best to using the phrase 'Pricing", "Website Pricing" or "Website Design Pricing" To me 'Pricing' is more appropriate because to the user they know they are on a website designer's site so what else would pricing be for right?! Furthermore you use less 'real estate' on the nav bar! There is on page text around the site which has links to "see our website design pricing" etc so I assume that is perhaps a more natural place to include that phrase? Look forward to your insights 🙂
On-Page Optimization | | NeilD0 -
Pages that show in Open Site Explorer that show as 404's in search
When I use Site Explorer to find our Top Pages there are a whole series of pages like this: http://www.nile-cruises-4u.co.uk/mybestbets/mybestbets-culture.html which I think were created when we had a Bulletin Board on the site. Although they are showing in the Top Pages results when you visit the page you get the following error message: HTTP Error 404.0 - Not Found The resource you are looking for has been removed, had its name changed, or is temporarily unavailable. What implications result from these pages still showing in results and what action should we take without messing up our search rankings? Any advice would be gratefully received.
On-Page Optimization | | NileCruises0 -
Value in a bottom navigation bar?
How useful is it to have links in the bottom navigation bar? I'm considering to putting a mini site map in the bottom navigation bar but was told that these links carry virtually no SEO value and could actually hamper it as it would bring the total number of links on the page over 100.
On-Page Optimization | | walidalsaqqaf0
