scaling up when new object is inserted in s3

Question

I want to setup auto scaling in following way.

• I am storing website data as zip in aws s3 bucket (versioning not enabled)
• when an instance is launched in auto scaling, user data will have shell script for fetching zip file from s3 bucket and unzip it to proper location.
• In future, when I have version 2 of zip file, auto scaling should stop older instances serving older website. And new instances should be spun up with new website (with same user data shell script).

how to achieve this?

Answer 1

I think you'll want to setup a CodeDeploy and CodePipeline to do this. Codepipeline can monitor the source, s3, and then send the update to codedeploy to update the EC2 instances.

http://docs.aws.amazon.com/codedeploy/latest/userguide/welcome.html

http://docs.aws.amazon.com/codepipeline/latest/userguide/welcome.html

Answer 2

I don't think this is possible using default auto scaling.

Auto scalings termination policy is:

1. Terminate instance in AZ with largest number of instances.
2. Oldest launch configuration
3. Closest to billing hour
4. Random

If you have one AZ the solution could be to change the launch configuration. Even if it's just a copy of the current one that's enough. That's not immediate though, that's eventually if things scale up and down regularly.

If you have multiple AZs a new launch configuration will work eventually.

You could manually protect instances from termination. That's manual though, and you'd have to change them on each zip file release.

Lambda gives you a lot of power and flexibility. You could have CloudWatch Events trigger a Lambda function. That function could check date of the latest zip file and dates of instance launch, and could terminate instances that are older than the zip file. This is probably the best automated way to do things.

Manual termination could possibly be easiest, so long as you don't release zip files too often.

Questions:

• How many AZs are you using?
• How often do you release zip files?