New "Static" Site with 302s
-
Hey all,
Came across a bit of an interesting challenge recently, one that I was hoping some of you might have had experience with!
We're currently in the process of a website rebuild, for which I'm really excited. The new site is using Markdown to create an entirely static site. Load-times are fantastic, and the code is clean. Life is good, apart from the 302s.
One of the weird quirks I've realized is that with oldschool, non-server-generated page content is that every page of the site is an Index.html file in a directory. The resulting in a www.website.com/page-title will 302 to www.website.com/page-title/.
My solution off the bat has been to just be super diligent and try to stay on top of the link profile and send lots of helpful emails to the staff reminding them about how to build links, but I know that even the best laid plans often fail.
Has anyone had a similar challenge with a static site and found a way to overcome it?
-
Wow. I wasn't expecting such a detailed and awesome answer Danny. Thanks so much, I'm in the process of migrating away from S3 anyways (for other reasons) though you're right in that I'm going to miss the cost & load times.
I'm using Middleman for now, though the technical part of my brain is indeed interested in how you're going to accomplish the Jekyll solution. I'll look out for your post!
And thanks for the tip on my site. Another thing to add to the list
Arun
-
Hey Arun,
Thanks for posting! I was beginning to think that I was the only Inbound guy anywhere that had to deal with this kind of issue
Yup, I created the same bug with redirect loops trying to get around the slash issue. The problem is that S3 doesn't consider the slash as part of the rewrite data unless something comes after it.
Ultimately, my number one suggestion would be to go with a different service that allows you to install a Server App like Nginx or Apache. Others have agreed that redirections set up through a server app are the way that they feel the most comfortable that link equity is being passed.
If you're dead-set on S3, which I would understand as the load times are crazy-awesome-insane, I may have a solution for you soon. Our dev team is working on a script for Jekyll + S3 sites that will essentially create extension-less files (i.e. example.com/contact) that contain meta refresh + rel canon.
The script will use a list of desired redirections + rules that is structured the same way an htaccess file would be. I can't speak to how it will get past S3's default 302ing yet, but I know that it will use CURL. Look for a YouMoz post soon from me!
Anyways, I hope my notes here help! I'm gonna try and make that post soon after the script is created. Just as a last note, in taking a look at your site I noticed that a lot of the internal links on your homepage don't have the trailing slash in them. I would definitely start there and add those slashes, and perform a "submit page + linked page" to Webmaster Tools after!
-
Hi Danny-
I've got the exact same issue (static site on S3 redirecting with 302s), and surprisingly can't find a lot of information out there. If I do a S3 metadata based redirect from (for example) /blog to /blog/ I just end up in a redirect loop.
I checked out your site and it still looks like you're working on it. Did you end up figuring anything out? If there's any way that I can help get to a solution I'd be happy to spend some time on it.
Thanks!
Arun
-
Thanks for the reply David!
Yup, I think that this has just been a case of wrapping my head around a new way of doing things (i.e. redirections in the AWS bucket config rather than using .htdocs). Static sites are a crazy combination of complicated and simple!
Thanks! We're using Jekyll somewhat, although we've had issues with the image hosting. I've actually had better results using the local github client + "Mou", a local Markdown editor.
-
Nice! (for speed at least)
I would show your team some examples of external URLs pointing at the non trailing slash versions of your pages and explain the downside of the 302 redirect. Also consider that people and bots visiting those URLs will be adding overhead to your server, and on Amazon that will equal increased cost (small as it may be, the pennies add up!)
Reading the link you provided it looks like the default behaviour of the page metadata redirect under the s3 console is to create a 301 redirect. That makes me think the 302 is coming from somewhere else. Look at the following URL:
http://docs.aws.amazon.com/AmazonS3/latest/dev/HowDoIWebsiteConfiguration.html
It looks like you can add advanced redirects under "Enable website hosting -> edit redirection rules". I'd explore if there are redirects listed there and maybe chat to your developers further.
While you are it I spotted two other issues for you to consider. Currently the index.html files in your directories resolve to the same page as your main directory. I would 301 those pages back to the parent directory (slash version). Or you could add canonical URLs pointing back to the parent directory (with trailing slash). I'd make a case for adding canonical URLs to all pages.
Also, you currently have a number of redirect chains e.g.
http://www.strutta.com/resources/posts/share-your-contests-and-sweepstakes-all-over-social-media 301 redirects to http://www.strutta.com/resources which 302 redirects to http://www.strutta.com/resources/.
You need to find the original redirect and change it to 301 redirect to the trailing slash version of the directory. Screaming Frog can help you find these redirect chains.
-
Hi Danny!
I don't have much to add here, I think the guys have it right in that you'll need to figure out how to make the 301 work. I quickly read that documentation, then realized I wasn't a robot, so I found this: http://aws.typepad.com/aws/2012/10/amazon-s3-support-for-website-redirects.html which was a bit more friendly.
I wish I could help you out more, but I'm not using AWS. I'm assuming you'll be able to use wildcard or regex matching somewhere, and that should solve your problem.
Great site by the way, anything you're using to help out with the static blog? (Jekyll, Octopress?)
-
Follow-up answer:
Our new website (Strutta.com) is entirely static, hosted on S3. No Apache, just straight HTML files. No apache means no htaccess.
Instead of using htaccess, we have to use the S3 Console: http://docs.aws.amazon.com/AmazonS3/latest/dev/how-to-page-redirect.html
As far as I can tell, this sets up redirects the same way. Although this doesn't answer my initial question, I'm going to try using the control panel later on today to see if 301ing the directories there to include the / will get recognized before whatever is causing the 302 currently
-
Thanks all,
I think the problem is coming from the fact that we're hosted on Amazon Webservices, and the devs are using the "aws bucket config" settings to institute redirects instead of htaccess. SEO vs Dev Battle time.
-
Hey Danny,
As Maximilian suggested above the best solution is going to be to change those 302s to 301s. I generally like to redirect to trailing slash URLs for directories and non trailing slash URLs for files/pages (that's that standard convention). I find in practice hardly anyone who links organically ever includes a trailing slash when linking to a page, but when it's the homepage I don't worry about it too much, browsers and Google can figure that out.
Basically you need to figure out where the 302 is coming from and hopefully it is in your .htaccess file. If you can edit your .htaccess file you need to change that to a 301 redirect, or you could remove the redirect and just use a canonical URL pointing at the / version of the page. I would prefer to go with the 301 though. Just be sure to look at how these redirects are being implemented and in what order, you don't want to end up with redirect chains either.
Can you get access to your .htaccess file or is the server running something funky?
-
Perhaps this is too obvious, but can you not change the 302 to 301's?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Webmaster Tools is saying "Sitemap contains urls which are blocked by robots.txt" after Https move...
Hi Everyone, I really don't see anything wrong with our robots.txt file after our https move that just happened, but Google says all URLs are blocked. The only change I know we need to make is changing the sitemap url to https. Anything you all see wrong with this robots.txt file? robots.txt This file is to prevent the crawling and indexing of certain parts of your site by web crawlers and spiders run by sites like Yahoo! and Google. By telling these "robots" where not to go on your site, you save bandwidth and server resources. This file will be ignored unless it is at the root of your host: Used: http://example.com/robots.txt Ignored: http://example.com/site/robots.txt For more information about the robots.txt standard, see: http://www.robotstxt.org/wc/robots.html For syntax checking, see: http://www.sxw.org.uk/computing/robots/check.html Website Sitemap Sitemap: http://www.bestpricenutrition.com/sitemap.xml Crawlers Setup User-agent: * Allowable Index Allow: /*?p=
Technical SEO | | vetofunk
Allow: /index.php/blog/
Allow: /catalog/seo_sitemap/category/ Directories Disallow: /404/
Disallow: /app/
Disallow: /cgi-bin/
Disallow: /downloader/
Disallow: /includes/
Disallow: /lib/
Disallow: /magento/
Disallow: /pkginfo/
Disallow: /report/
Disallow: /stats/
Disallow: /var/ Paths (clean URLs) Disallow: /index.php/
Disallow: /catalog/product_compare/
Disallow: /catalog/category/view/
Disallow: /catalog/product/view/
Disallow: /catalogsearch/
Disallow: /checkout/
Disallow: /control/
Disallow: /contacts/
Disallow: /customer/
Disallow: /customize/
Disallow: /newsletter/
Disallow: /poll/
Disallow: /review/
Disallow: /sendfriend/
Disallow: /tag/
Disallow: /wishlist/
Disallow: /aitmanufacturers/index/view/
Disallow: /blog/tag/
Disallow: /advancedreviews/abuse/reportajax/
Disallow: /advancedreviews/ajaxproduct/
Disallow: /advancedreviews/proscons/checkbyproscons/
Disallow: /catalog/product/gallery/
Disallow: /productquestions/index/ajaxform/ Files Disallow: /cron.php
Disallow: /cron.sh
Disallow: /error_log
Disallow: /install.php
Disallow: /LICENSE.html
Disallow: /LICENSE.txt
Disallow: /LICENSE_AFL.txt
Disallow: /STATUS.txt Paths (no clean URLs) Disallow: /.php$
Disallow: /?SID=
disallow: /?cat=
disallow: /?price=
disallow: /?flavor=
disallow: /?dir=
disallow: /?mode=
disallow: /?list=
disallow: /?limit=5
disallow: /?limit=10
disallow: /?limit=15
disallow: /?limit=20
disallow: /*?limit=250 -
Handling "legitimate" duplicate content in an online shop.
The scenario: Online shop selling consumables for machinery. Consumable range A (CA) contains consumables w, x, y, z. The individual consumables are not a problem, it is the consumables groups I'm having problems with. The Problem: Several machines use the same range of consumables. i.e. Machine A (MA) consumables page contains the list (CA) with the contents w,x,y,z. Machine B (MB) consumables page contains exactly the same list (CA) with contents w,x,y,z. Machine A page = Machine B page = Consumables range A page Some people will search Google for the consumables by the range name (CA). Most people will search by individual machine (MA Consumables, MB Consumables etc). If I use canonical tags on the Machine consumable pages (MA + MB) pointing to the consumables range page (CA) then I'm never going to rank for the Machine pages which would represent a huge potential loss of search traffic. However, if I don't use canonical tags then all the pages get slammed as duplicate content. For somebody that owns machine A, then a page titled "Machine A consumables" with the list of consumables is exactly what they are looking for and it makes sense to serve it to them in that format. However, For somebody who owns machine B, then it only makes sense for the page to be titled "Machine B consumables" even though the content is exactly the same. The Question: What is the best way to handle this from both a user and search engine perspective?
Technical SEO | | Serpstone0 -
How can I change the page title "two" (artigos/page/2.html) in each category ?
I have some categories and photo galleries that have more than one page (i.e.: http://www.buffetdomicilio.com/category/artigos and http://www.buffetdomicilio.com/category/artigos/page/2). I think that I must change the tittle and description, but I don't how. I would like to know how can I change the title of each of them without stay with duplicate title and description. Thank you! ahcAORR.jpg
Technical SEO | | otimizador20130 -
Accidentally checked privacy setting in WP to "not to index" and dropped rank...how can I fix this?
I recently rebuilt a static website to a wordpress site...In the privacy settings ....the -"Ask search engines not to index this site" was checked and I didn't notice. I had a top ranking website now its completely gone off google and every where else. I have unchecked it, resubmitted a sitemap to google.....does anyone know if this is permanent damage or if there is something else I can do to help fix this......I'm freaking out
Technical SEO | | eversseo0 -
How valuable is content "hidden" behind a JavaScript dropdown really?
I've come across a method implemented by some SEO agencies to fill up pages with somehow relevant text and hide it behind a javascript dropdown. Does Google fall for such cheap tricks? You can see this method used on these pages for example (just scroll down to the bottom) - it's all in German, but you get the idea I guess: http://www.insider-boersenbrief.de/ http://www.deko-und-kerzenshop.de/ How is you experience with this way of adding content to a site? Do you think it is valuable or will it get penalised?
Technical SEO | | jfkorn0 -
Moz Crawl Reporting Duplicate content on "template" styled pages
We have a lot of detail pages on our site that reference specific scholarships. Each page has a different Title and Description. They also have unique information all regarding the same data points. The pages are displayed in a similar structure to the user so the data is easy to read. My problem is a lot of these pages are being reported as duplicate content when they certainly are not. Most of them are reported as duplicates when they have the same sponsor. They may have the same contact information listed. These two are being reported as duplicate of each other. They share some data but they are definitely different scholarships. http://www.collegexpress.com/scholarships/adelaide-mcclelland-garden-club-scholarship/9254/ http://www.collegexpress.com/scholarships/mary-wannamaker-witt-and-lee-hampton-witt-memorial-scholarship/10785/ Would it help to add a Canonical for each page to themselves? Any other suggestions would be great. Thanks
Technical SEO | | GeorgeLaRochelle0 -
"Spam emails" : ranking drop?
Hello, Is it possible that a website gets penalised by Google because your hosting company blocked you from sending emails? Basically I got a message from my hosting company saying that they were blocking me from sending emails from our server and domain because too many had mistakes or were complained about. The same day we dropped from 2<sup>nd</sup> on a keyword to about 600<sup>th</sup> while still being ranked for other keywords. The drop was for our main keyword. Can the fact we sent “bad emails” be related to a rank drop? For the record, those were confiormation emails for account creation, they were legit, not spam. That's off-topic though.
Technical SEO | | EndeR-0 -
New website with slightly new urls
Hi we recently designed our website in work and changed some of the urls. the old site used to be http://www.example.ie/contact-us.htm now it's is http://example.ie/get-in-touch The problem we are having is with sitelinks (the ones auto generate in the serp) ie: about, contact us, team etc etc. Once cliked on, these OLD links are all going to 404 pages because of the change of url. Help with this would be greatly appreciated - I was thinking of blocking these old sitelinks in google web master.
Technical SEO | | GlenBOB0