7

I have a basic-ish ELB v2 site. No clustering or anything yet. I'm pretty inexperienced with AWS.

My stack is nginx/uwsgi/django + some other services.

I was wondering if anyone had thoughts about making a "sorry, the website is currently down..."-style page (custom text I can update planned downtime is a bonus!) whenever there is downtime for whatever reason, and the health of the instance is red. It doesn't seem that Amazon offers this capability - am I missing anything? Is there a way to create a separate, super-tiny instance that is ONLY served if the main one is red, or something?

Thanks!

19

The simple and cool solution here is to put your ELB behind CloudFront.

If the origin server (the ELB in this case) throws a 5XX error (or 4XX if you like), CloudFront can return a custom error page, which you can configure CloudFront to fetch from an S3 bucket by creating a second origin pointing to the bucket and creating a cache behavior routing (e.g.) /errors/static/* to the bucket.

This works better than Route 53 failover for an important reason... a fatal flaw, if you will... browsers are terrible about caching DNS lookups for far longer than you expect. The DNS TTL isn't relevant.

Essentially, once a browser has a DNS entry in hand, it just keeps trying to use it... typically, until all browser windows are closed.

So if your site goes down for a visitor who was already on the site, they unlikely to see the alternate site.

Worse, if a visitor hits your site for the first time while it's down, they'll "stick" on the maintenance page until they close all browser windows.

If you use failover DNS, that's really only good if the failover target is still your application, maybe just further away.

You can turn CloudFront's caching off if you don't need it.

You can also configure CloudFront's error caching TTL to a nonzero value if you want it to quit hammering your site while it's down and trying to recover. For a given page that throws an error, it will keep showing the error page and not bother your server with more requests for that page until the Error CachingTTL expires.

  • Interesting. AWS don't mention this drawback in their documentation. This underlines how important testing is. – Tim Jan 25 '17 at 1:24
  • Well @Tim, AWS recommends doing rolling updates. They didn't have their "Docker Service" that they offer now, so EBS it was for our Docker app. Only needed one, though. – std''OrgnlDave Jul 28 '17 at 21:52
6

Use Route53 DNS and failover routing. You should be able to get up an S3 bucket hosting a single page static website. I don't think you can do it with just ELB.

Amazon have a blog post that tells you how to do it here.

Update: as Michael says, there's a drawback around browser DNS caching, see his answer for more information. Route 53 is probably a simpler option than CloudFront, but CF has other advantages, performance and in some use cases can lower costs.

  • I do say +1 here... this is a good solution, but it does seem to have an achilles heel, making it less viable in some use cases. – Michael - sqlbot Jan 25 '17 at 1:14
  • @Michael-sqlbot what's the drawback of this approach? Browser DNS caching time? – Tim Jan 25 '17 at 1:22
  • Yes, exactly. People "sticking" to the error page is something I find unnerving. – Michael - sqlbot Jan 25 '17 at 5:09
  • Here is an updated blog post from AWS with an newer simpler method for ELB failover routing. aws.amazon.com/blogs/aws/… See my post below for more details. – AstroTom Jul 24 '18 at 12:43
1

Several solutions have been mentioned already, including CloudFront and Route53. CloudFront is an excellent solution and in my experience hasn't slowed things down at all, but it does bring added cost. And Route53 has the DNS caching issues already mentioned.

Until ALB supports custom error pages out of the box (which may or may not happen), there's potentially a new solution after the recent announcement of ALB fixed responses, but it's not point and click: you could set up a Lambda function that temporarily adds a rule to your load balancer, providing a fixed response with your 'error page' contents.

Aside from writing the Lambda, you'll need to find a way to trigger it 'on' and 'off', which could possibly be via a Route53 health check or load balancer target group health check (probably via CloudWatch alarm -> SNS -> Lambda).

It's not exactly simple, but would probably work well once set up!

0

As written by @Tim and @Micheal, you have your choice of using Route53 DNS and failover routing, or CloudFront with custom error pages. Both methods have their pros and cons.

If you are not already using Cloudfront, I think Route53 is a simpler solution. See the updated blog post from AWS (which now includes a simpler method for ELB integration).

CloudFront is much more complicated to setup, and it will take about 15 minutes for every update. Cloudfront also caches (as to be expected) so it is not clear if that will be any slower to respond, than the DNS cache issues with Route 53.

Note that if your ELB website only responds to SSL then you cannot use the simple S3 solution as described in the AWS blog3. In that case you will have to add Cloudfront in front the the S3 bucket to add the SSL, this making the DNS failure routing solution more complicated.

  • That blog post does not offer a clean solution -- it has exactly the problem I mentioned in my post and discussed with Tim in comments. It is perfectly viable when you have multiple deployments that can serve your requests but wholly unsuited to an error page because of the way browsers cache DNS lookups. The content of the AWS post unfortunately fails to take this reality into consideration. DNS failover does not reliably "fail back" from the end-user's perspective. CloudFront also has entirely separate cache settings for error responses. – Michael - sqlbot Sep 28 '18 at 7:45

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.