Pitfalls when implementing the “VARY User-Agent” server response
-
We serve up different desktop/mobile optimized html on the same URL, based on a visitor’s device type.
While Google continue to recommend the HTTP Vary: User-Agent header for mobile specific versions of the page (http://www.youtube.com/watch?v=va6qtaiZRHg), we’re also aware of issues raised around CDN caching; http://searchengineland.com/mobile-site-configuration-the-varies-header-for-enterprise-seo-163004 / http://searchenginewatch.com/article/2249533/How-Googles-Mobile-Best-Practices-Can-Slow-Your-Site-Down / http://orcaman.blogspot.com/2013/08/cdn-caching-problems-vary-user-agent.html
As this is primarily for Google's benefit, it's been proposed that we only returning the Vary: User-Agent header when a Google user agent is detected (Googlebot/MobileBot/AdBot).
So here's the thing: as the server header response is not “content” per se I think this could be an okay solution, though wanted to throw it out there to the esteemed Moz community and get some additional feedback.
You guys see any issues/problems with implementing this solution?
Cheers!
linklater
-
So, there are lots of 'ifs' here, but the primary problem I see with your plan is that the CDN will return the content to Googlebot without the request hitting your server so you won't have the option to serve different headers to Googlebot.
Remember that every page is the main HTML content (which may be static or dynamically generated for every request), and then a whole bunch of other resources (Javascript and CSS files, images, font files etc.). These other resources are typically static and lend themselves far better to being cached.
Are your pages static or dynamic? If they are dynamic then you are possibly not benefitting from them being cached anyway, so you could use the 'vary' header on just these pages, and not on any static resources. This would ensure your static resources are cached by your CDN and give you a lot of the benefit of the CDN, and only the dynamic HTML content is served directly from the server.
If most of your pages are static you could still use this approach, but just without the full benefit of the CDN, which sucks.
Some of the CDNs are already working on this (see http://www.computerworld.com/s/article/9225343/Akamai_eyes_acceleration_boost_for_mobile_content and http://orcaman.blogspot.co.uk/2013/08/cdn-caching-problems-vary-user-agent.html) to try and find better solutions.
I hope some of this helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Proxy Servers & SEO
Does putting a blog on a proxy server (the pointed at the main site) hurt SEO? i.e. can Google tell? And if they can, does it matter? My server people won't use PHP on their servers but we want a Wordpress blog. So their suggested solution is that they put the blog on a proxy server and point it at the ourdomain.com/blog subfolder on our site. So to all intents and purposes it's hosted in the same place. They assure me this is normal practice and point out that our (main site) images are already being sourced from a CDN. Obviously we'll deal with Google not seeing two separate versions of the same site. But apart from this, is there any negative effect we could suffer from in SEO terms?
Intermediate & Advanced SEO | | abisti20 -
Fast/Easy Way to Implement Canonical tags in Bulk in Magento CMS?
Hello Amazing SEO Community! Quick Q for a client with a TON of duplicate content. (yikes!) My client is currently undertaking a large SEO project around canonical tagging for their thousands of duplicate pages. Currently, one product sits on multiple URLs and they are being indexed as different pages (with the same content). The issue is found across all products and other pages, and across their international sites as well. One core challenge they face now is lack of time/resources from their developer side. The solution we see to the duplicate content is to manually add a canonical tag to each of our tens of thousands of pages. Their content management system is Magento. Has anyone ever tackled canonicalization for a large site that uses Magento? Any more efficient solutions to manual tagging is ideal. Thanks in advance for your input. -Bonnie
Intermediate & Advanced SEO | | accpar0 -
How does Tripadviser ensure all their user reviews get crawled?
Tripadvisor has a LOT of user generated content. Searching for a random hotel always seems to return a paginated list of 90+ pages. However once the first page is clicked and "#REVIEWS" is appended to the URL, the URL never changes with any subsequent clicks of the paginated links. How do they ensure that all this review content gets crawled? Thanks, linklater
Intermediate & Advanced SEO | | linklater0 -
SEO Concerns From Moving Mobile M Dot site to Responsive Version?
I currently have my mobile site set up as a m dot site. I have designed a new responsive/adaptive version of my desktop site I would like to start using. When I search from google on mobile, my website is indexed as the m dot site. When I make the switch, this will no longer be the case as I will only have one url for both mobile and desktop. The m dot url's will no longer work. Are there any SEO consequences from making this shift?
Intermediate & Advanced SEO | | mikeylong70 -
How to implement three languages with a subfolder /blog on a .com domain?
Hi, I'm setting up a blog for a client that has a .com domain. The client is targeting three languages with the subfolder /de, /nl and /en. We've also established that a blog rather be used in a subfolder than subdomain. My question: How should we implement the three languages? Is this gonna be a domain.com/blog/en or domain.com/en/blog or would you maybe don't use subfolders for language at all but let a hreflang do the job?
Intermediate & Advanced SEO | | dexport0 -
Implementing Large-Scale Redirects
Hello All, I have a situation with my site where a vendor created a local directory of locations on a sub-domain of my site. This sub-domain has approximately 2000 pages. It is a PR3 and a good backlink profile (not many links. Mostly citations. Not spammed). It get decent traffic but 80% of the traffic is driven by ppc. We have created a new local section on the main page of our website and we are trying to weigh the benefit of redirecting all of those pages on the old sub-domain. We anticipate that this new section will begin to replace the old sub-domain in serps. Additionally, when our deal with the company that manages this sub-domain ends in three months, the pages will no longer exist. Is it worth redirecting the pages (you might need more information to give good insight into that)? Also, if we do implement approx. 2000 redirects, what effect will that have on the main site from an SEO perspective. Is it possible that Google might ignore this large scale redirect effort? Will the value also be limited by the fact the redirect might only be live for a month before the original pages are deleted? Any help/insight with this would be greatly appreciated. Thanks!
Intermediate & Advanced SEO | | ResponseMine0 -
How important is it to fix Server Errors?
I know it is important to fix server errors. We are trying to figure out how important because after our last build we have over 19,646 of them and since google only gives us a 1000 at a time the fastest way to tell them we have fixed them all is to use the api etc which will take time. WE are trying to decide is it more important to fix all these errors right now or focus on other issues and fix these errors when we have time, they are mostly ajax errors. Could this hurt our rankings? Any thoughts would be great!
Intermediate & Advanced SEO | | DoRM0 -
Multilingual sites: Canonical and Alternate tag implementation question
Hello, I would like some clarification about the correct implementation of the rel="alternate" tag and the canonical tag. The example given at http://support.google.com/webmasters/bin/answer.py?hl=en&answer=189077 recommends implementing the canonical tag on all region specific sub-domains, and have it point to the www version of the website Here's the example given by Google. My question is the following. Would this technique also apply if I have region specific sites site local TLD. In other words, if I have www.example.com, www.example.co.uk, www.example.ca – all with the same content in English, but prices and delivery options tailored for US, UK and Canada residents, should I go ahead and implement the canonical tag and alternate tag as follows: I am a bit concerned about canonicalizing an entire local TLD to the .com site.
Intermediate & Advanced SEO | | Amiee0