Is the If-Modified-Since HTTP Header still relevant?
-
I'm relatively new to the technical side of SEO and have been trying to brush up my skills by going through Google's online Web-master Academy, which suggests that you need a If-Modified-Since HTTP Header tag on your site. I checked and apparently our web server doesn't support this.
I've been told by a good colleague that the If-Modified-Since tag is no longer relevant as the spiders will frequently revisit a site as long as you regularly update and refresh the content (which we do).
However our site doesn't seem to of been reindexed for a while as the cached version's are still showing the pages from over a month ago.
So two question really - is the If-Modified-Since HTTP Header still relevant and should I make sure this is included?
And is there anything else I should be doing to make sure the spiders crawl our pages? (apart from keeping them nice, fresh and useful)
-
If the webserver does not support (or the admin does not want to enable) this feature you could always have your frontend-templates have a small string wich holds the date/time when the page was last updated. Something along the lines "last updated on: ...." at the bottom or top of the content area. It's also an useful bit of information for users.
-
Hi Annie
I'm surprised there hasn't been lots of answers to your question.
Check-out this video here on SEOmoz entitled "Whiteboard Interview - Google's Matt Cutts on Redirects, Trust + More" featuring Matt Cutts being asked some questions by Rand. It opens with a partial answer to your first question:
"These days we use it a little less" (2 years ago) ~ basically means that in locations such as the US, most of Europe, Japan... & so on, where Bandwidth is rarely an issue anymore, 'If-Modified-Since' isn't taken notice of, it's not worth including anymore.
In say developing countries where bandwidth is sometimes still on the low side, it may still be used, hence why a sweeping 'it doesn't matter anymore' statement wasn't given.
**Your second question: **
- Content, fresh unique value-adding content that is, that's engaging and shareable, is always a positive aspect to work on, which in turn can lead to some awesome new links. This encourages the bots to visit more regularly.
- Ensuring that your site doesn't have any technical issues (say causing significant downtime).
- Ensuring that Robots.txt isn't wrongly disallowing any pages from being crawled.
- Keeping an eye on Google Webmaster Tools (& Bing Webmaster Tools) for any messages or errors.
- You can alter the crawl rate in GWT, though is usually best to leave it on the default auto setting.
Hope that helps,
Simon
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Https to http
Two versions of the same page are being served at the moment. Certain pages on the site redirect to the https version whilst others don't. I am being flagged for duplicate content because of this. Is this a simple fix? As in just redirect all to the https version and set the preferred version in WMT?
Technical SEO | | TAT1000 -
Secure HTTP Change - No Links in WMT
Website was changed over to secure HTTP about twp months ago. Just looked in Google Webmaster Tools and it only shows about 8 inbound links. We did a permanent 301 redirect for all URLs. There are over 800 links according to Open Site Explorer. Is it just that they are showing only the HTTPS inbound links? Should I add the HTTPS version in WMT? Thanks for any assistance
Technical SEO | | EBI0 -
Why seomoz.org still in Google index?
I searched in Google, the number of URLs indexed left in the seomoz.org domain since it changed to moz.comI am surprised that after all this time more than 15,000 URLs indexed:https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=site%3Aseomoz.org%20inurl%3Aseomoz.org If I clicked on any of the results it will be redirect (301) to the new domain, so it is working, but Google still keep these URLs in the index.
Technical SEO | | Yosef
What could be the reason?Will not cause duplicated content issue on moz.com?0 -
Http to Https Backlink Value
We updated our website from http to https and I wanted to know how backlinks were affected by this. The new site redirects all old http links to the https home page. How does this affect more specific backlinks like http://www.mysite.com/about ? The old http://www.mysite.com/about is now being directed to https://www.mysite.com . Do I need to set up redirects to pass value to my new pages?
Technical SEO | | nat88han0 -
PR / News stories across multiple sites - is it still duplicate content?
I was wondering does Google make an exception for news stories where duplicate content is concerned? After all depending on the story there can be a lot of quotes and bulk blocks of the same details. Is Google intelligent enough to distinguish between general website content and actual news stories? Also like a lot of big firms we publish news stories on our website, but then they get passed on to other websites in the form of PR, and then published on other websites. So if we put it on our website, then within a few hours or the same day other websites publish the story at the same time (literally copied and pasted) - how does this affect our website in terms of duplicate content? Will Google know automatically that we published it first? Thanks!
Technical SEO | | Brabian0 -
Looking for live website examples of google adsense since panda update
Hi, i am looking for some live websites that use google adsense who have changed their layout since the panda update If anyone can please point me in the right direction so i can see good examples of how sites have now changed their google adsense layout so we can learn from this, then that would be great regards
Technical SEO | | ClaireH-1848860 -
Round 3 & still no indexing for varicose veins :-(
Greetings from 11 degrees C partly suuny Wetherby 🙂 Every so oftem you hit an SEO mission that just consistently hits a brick wall. For the third time i'm investigating why this page:
Technical SEO | | Nightwing
http://www.collegeofphlebology.com/varicose-veins/what-are-they/ fails to even reach the bottom of page 3. Ive gone back to basic and ran an SEO audit of sorts in an attempt to see if I'd missed anything. Here is the audit: http://i216.photobucket.com/albums/cc53/zymurgy_bucket/audit-for-moz.jpg So my question is please: From a technical SEO perspective is there anything wrong with this page http://www.collegeofphlebology.com/varicose-veins/what-are-they/ to explain why it does not rank for target term "Varicose Veins" Thanks in advance,
David0 -
Another http vs https Question?
Is it better to keep the Transaction/ Payment pages on a commercial website as the only secure ones (https) and remainder of website as http? Or is it better to have all the commercial website as secure (https)?
Technical SEO | | sherohass0