2

I am trying to setup auto-healing for my EC2 instance. So whenever my instance goes down or is unreachable, it should auto reboot or launch a new instance preferrably in a different AZ in the same AWS region.

I looked into setting up cloudwatch alarm and adding action to reboot the instance in case of failure. This works well if I want my EC2 instance to just reboot in case of failure.

But it doesn't cover the case where AZ itself is not reachable. So I am trying to setup AWS autoscaling group with min/max instance number as 1. In case of failure I want new instance to be launched with EBS volume of terminated instance. How do I go ahead with this?

Or if there is an alternate way to achieve what I want, that would be appreciated too.

  • 1
    If the availability zone is unreachable, the volume is unreachable. EBS is only replicated within availability zones, not across them. Also, what if the old instance failed because of a problem with the volume? An "alternate way to achieve" what you want? What is it that you really want? If it's cross-AZ availability of static data then you likely want EFS. – Michael - sqlbot Sep 12 '17 at 10:50
  • There's of course the relatively obvious, simple solution of taking a snapshot of the volume. Snapshots are stored in S3, which is replicated across AZs. You're likely to be able to access the S3 snapshot if any single AZ is down, and create a volume from that snapshot. – Tim Sep 12 '17 at 18:51
1

So I run a lot of single servers in ASGs for auto-healing.

What I did was give the EBS volume a set tag.

On boot there's a cloud-init script that uses the aws meta data api and CLI and gets its own instance ID via the API and then searches for a EBS volume with a set tag in its own AZ. Once it has the volume ID it forces a detach to be on the safe side. Then attaches it to itself.

So far its worked pretty well.

0

Do you know about EFS or Amazon Elastic File System? It is a file storage service for EC2 instances and you can do with it what you described. If you don't necessary need a block device (EBS volume) give it a try.

There is a nice example of CloudFormation template which deploys EC2 instances in an AutoScaling group that are associated with an Amazon EFS file system: Amazon Elastic File System Sample Template

0

This is something that could be done, but i don't think it would be easy.

You'll need to use a secondary volume for whatever data you wish to persist across instance restarts. I don't believe you can create an instance with an existing volume as the root partition. Not in any sane or easy way at least.

Set EBS "Delete on Termination" to false, and have your launch configuration/startup script scan for un-attached volumes on startup. If you find your volume tagged "KeepMeAlive:true" or something similar, have it automatically attach that volume to itself.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.