Is the If-Modified-Since HTTP Header still relevant?
-
I'm relatively new to the technical side of SEO and have been trying to brush up my skills by going through Google's online Web-master Academy, which suggests that you need a If-Modified-Since HTTP Header tag on your site. I checked and apparently our web server doesn't support this.
I've been told by a good colleague that the If-Modified-Since tag is no longer relevant as the spiders will frequently revisit a site as long as you regularly update and refresh the content (which we do).
However our site doesn't seem to of been reindexed for a while as the cached version's are still showing the pages from over a month ago.
So two question really - is the If-Modified-Since HTTP Header still relevant and should I make sure this is included?
And is there anything else I should be doing to make sure the spiders crawl our pages? (apart from keeping them nice, fresh and useful)
-
If the webserver does not support (or the admin does not want to enable) this feature you could always have your frontend-templates have a small string wich holds the date/time when the page was last updated. Something along the lines "last updated on: ...." at the bottom or top of the content area. It's also an useful bit of information for users.
-
Hi Annie
I'm surprised there hasn't been lots of answers to your question.
Check-out this video here on SEOmoz entitled "Whiteboard Interview - Google's Matt Cutts on Redirects, Trust + More" featuring Matt Cutts being asked some questions by Rand. It opens with a partial answer to your first question:
"These days we use it a little less" (2 years ago) ~ basically means that in locations such as the US, most of Europe, Japan... & so on, where Bandwidth is rarely an issue anymore, 'If-Modified-Since' isn't taken notice of, it's not worth including anymore.
In say developing countries where bandwidth is sometimes still on the low side, it may still be used, hence why a sweeping 'it doesn't matter anymore' statement wasn't given.
**Your second question: **
- Content, fresh unique value-adding content that is, that's engaging and shareable, is always a positive aspect to work on, which in turn can lead to some awesome new links. This encourages the bots to visit more regularly.
- Ensuring that your site doesn't have any technical issues (say causing significant downtime).
- Ensuring that Robots.txt isn't wrongly disallowing any pages from being crawled.
- Keeping an eye on Google Webmaster Tools (& Bing Webmaster Tools) for any messages or errors.
- You can alter the crawl rate in GWT, though is usually best to leave it on the default auto setting.
Hope that helps,
Simon
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How can I stop a tracking link from being indexed while still passing link equity?
I have a marketing campaign landing page and it uses a tracking URL to track clicks. The tracking links look something like this: http://this-is-the-origin-url.com/clkn/http/destination-url.com/ The problem is that Google is indexing these links as pages in the SERPs. Of course when they get indexed and then clicked, they show a 400 error because the /clkn/ link doesn't represent an actual page with content on it. The tracking link is set up to instantly 301 redirect to http://destination-url.com. Right now my dev team has blocked these links from crawlers by adding Disallow: /clkn/ in the robots.txt file, however, this blocks the flow of link equity to the destination page. How can I stop these links from being indexed without blocking the flow of link equity to the destination URL?
Technical SEO | | UnbounceVan0 -
Working out whether a site is http and https
Hi there, I can access the following site with http and https making me think that there will be a duplicate content issue. How can I work out if this is the case? http://ionadebarge.com https://ionadebarge.com Thanks.
Technical SEO | | Bee1591 -
Https vs http two different domains?
If i visit mywebsite.com.au, www.mywebsite.com.au and http://www.mywebsite.com.au - i get one website BUT if I visit https://www.mywebsite.com.au I get a different website - I also get a untrusted website warning The logo in the bottom right of the https: website is the name of the webdesigner where the website is hosted. Is this a normal practice?
Technical SEO | | GardenBeet0 -
Rel="canonical" What if there is no header??
Hi Everyone! Thanks to moz.com, I just found out that we have a duplicate content issue: mywebsite.com and mywebsite.com/index.php have the same content. I would like to make mywebsite.com the main one because it already has a few links and a better page rank. I know how to do a 301 redirect (already have one for www.mywebsite.com) but I am aware that a 301 redirect for my index file would create a loop issue. I have read the article about redirecting without creating a loop (http://moz.com/blog/apache-redirect-an-index-file-to-your-domain-without-looping) but quite frankly I don't even have a clue what he's trying to tell me (e.g. "Create an apache DirectoryIndex directive for your document root." What????!)… So I figured a rel="canonical" tag for my index file would be easier and fix the problem, too (right??) In every "How to" description they always say you have to put the rel="canonical" tag in the header of your duplicate content file. But: My index.php has no header (or nothing that looks like a header to me)! This is what it looks like: foreach($_GET as $key => $value)
Technical SEO | | momof4
{
$$key = $value;
}
foreach($_POST as $key => $value)
{
$$key = $value;
}
$page_title="my title";
$page_description="my description";
$page_keywords="keywords";
//echo $link;
//exit;
if (!isset($link)):
$page_content="homepage.php";
else:
if ($link=="services"):
$page_content="services.php";
$page_title=" my title for services page";
$page_description="description for services.";
endif;
… ect. for the other pages So where do I put the rel=canonical tag? Or is there another solution for the whole problem? Like delete the whole index file (lol) Thanks in advance for any answers!0 -
Top 10 keywords is still going strong but the rest just got smashed!
Hi SEOMOZ and it's USERs, Been trying to find my answer online but now after three weeks of reading blogposts I'm going to try this. 🙂 My website was ranking really good on 10 important keywords but not so good on the long tail, between 11 - 50 on maybe 30 different other, not so important keywords.
Technical SEO | | Drillo
So I began doing some work (I'm a newbie) but this is what I did:
1. Changed top navigation structure to get 4 other pages (w/ keywords as links) in it. (used a dropdown)
2. Wrote plenty of text that was a good fit for the page. (The text is OK and not to spammy looking.)
3. Added three links from high quality sites with keywords as links to these pages. I added them from my own site that is on the same server, same IP. 😉 I know, not looking so good.
4. Changed URL structure on a couple of pages to get a keyword in it. (did a correct 301)
5. Changed to better Titles and headings on the page. Keywords in them both but not the same. The result:
1. My 10 most important keywords I began ranking even better. I rank no. 1 on 9 out of 10.
2. Almost all the other pages went from ranking ~ 15 - 50 to not > 50. It has now been 4 weeks since I did most of the changes and 3 weeks since all the pages was hit > 50. So now I'm thinking about what to do?
1. Should I clean up my text, titles so they don't look to over optimized?
2. Should I remove my links from my own pages? (my link profile in general is actually pretty good.)
3. or should I just wait? Because changing more will just indicate to Google that somehing fishy is going on 😉 ? In the beginning I hoped that Google killed my rankings just because of the big changes. But now after 3 weeks I'm more sceptical and thinks I've been hit by a over-optimizing filter. According to webmaster tools I've not been hit by a manually penalty. Please, help me. I would really appreciate all ideas from people here with more experience.0 -
A while back there was a strategy presented about developing games or widgets and then using them for link bait. Is this still a viable strategy to improve ranking?
We are developing a game (might be an app or a widget or both) and are looking for the latest advice about how to leverage it for seo benefit. Can anyone share their recent experience with this? I know it was a strategy used in the past, but what considerations should we be looking at (technically and strategically) as we begin developing. Thanks in advance for any advice.
Technical SEO | | larahill0 -
Url canonicalization: www. to http://
Hey there. Sorry for the simple question but I recently redesigned a site and published with WordPress, in the process the domain structure changed from being www. to http:// . My question is does this change affect the value we get from links pointing to the old www. domain structure? The reason I ask is that the old site had a domain authority of 36 with OSE and a couple of hundred links but the new site address shows as having zero domain authority and zero links. Is there some best practise I should be following to retain link value?
Technical SEO | | Luia0 -
Best place for new relevant content ?
Hi everyone, The background to my question is that one of the companies I work for has just produced a load of videos answering many of the common questions about their products, for the ecommerce part of their site. I've got these videos transcribed which I want to use as the backbone to some of the main pages on the site (some of which are already ranking reasonably well). I was thinking of putting this content on their department page (so for example their departments would be kitesurfing, land kites, toy kites etc.) above links to their sections (e.g. kite surf kites, kite surf boards etc.). Do you think this would be a good place to put a large amount of text ? (the text will be in a scrollable div) - i.e. could it get in the way of the links which are currently passing juice deeper in to the site ? Alternatives are to put the videos under the links to the sections (which would place them below the fold), or to place each video on its own page and simply link to these pages from the departments page (although it is the department page we wish to rank). Another alternative is to put links to the sections both before and after the new content. Hope that all makes sense! Thanks in advance
Technical SEO | | stukerr0