Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am setting up auto scaling based on memory consumed by the particular instance. I went through the link posted at awsforums. I was able to set my metrics and using the bash script given and with the same variables.

In figuring out how to set auto scaling based on those metrics, I came to know about the bunch of bash scripts at google project hosting site. I am not able to set the metrics.

How could I move further?

Image showing 4 Cloudwatch alarms do not have enough data to be evaluated

share|improve this question
1  
If I understand your question correctly - the metrics you set from the forum script show up in Cloudfront, while the ones set from the Google Code project do not. A quick glance at the Google Code project suggests that the environment variables are not exported - copy the working variable definitions (the export lines) from the top of forum script to ec2-write-memory-metrics.sh. –  cyberx86 Jan 12 '12 at 10:43
    
@cyberx86 I was able to run the script specified in the forum, when I type the mon-list-metrics I am getting the metrics details and its been executed as a cron job also its working fine, what are the further steps I need to carry out to accomplish the auto scaling based on memory-metrics. –  Jeevan Dongre Jan 12 '12 at 11:10
    
Maybe the answers of this similar question will help you: serverfault.com/questions/348345/… –  Daniel Dener Mar 2 '13 at 19:21
    
Maybe the answers of this similar question will help you: serverfault.com/questions/348345/… –  Daniel Dener Mar 2 '13 at 19:22

2 Answers 2

up vote 3 down vote accepted

Once you have the memory metrics logged to Cloudwatch, you simply need to setup autoscaling as you would for any existing metric.

Firstly, as with all the AWS command line tools, you need to set (export) either:

  • AWS_CREDENTIAL_FILE or
  • both: EC2_PRIVATE_KEY and EC2_CERT

Next you run the following three commands, modifying them as per your needs (these are just run from the command line - once - not from cron).

Create the launch config: You need to pass an image and instance type at very least, additional parameters are optional, but probably a good idea.

as-create-launch-config
        LaunchConfigurationName  --image-id  value  --instance-type  value
       [--block-device-mapping  "key1=value1,key2=value2..." ] [--kernel  value]
       [--key  value ] [--ramdisk  value ] [--group  value[,value...] ]
       [--user-data  value ] [--user-data-file  value ]  [General Options]

For example:

as-create-launch-config config-name --image-id AMI-xxxxxxxx --instance-type m1.small --key keypair-name --group security-group-name

Create the autoscaling group: Here we define the parameters for scaling - where the instances will be launched, the limits on the number of instances, and associate the group with the config we created.

as-create-auto-scaling-group
        AutoScalingGroupName  --availability-zones  value[,value...]
        --launch-configuration  value  --max-size  value  --min-size  value
       [--cooldown  value ] [--load-balancers  value[,value...] ]
        [General Options]

For example:

as-create-auto-scaling-group as-group-name --availability-zones us-east-1a --launch-configuration config-name --min-size 1 --max-size 5 --cooldown 300

(You can also specify a --desired-capacity which is the number of instances to start with, if it is different than the min-size)

Create a policy to scale with: Here we define the action that will be performed when an alarm is triggered and associate the policy with a specific autoscaling group. Create one policy for scaling up, and one for scaling down.

as-put-scaling-policy
        PolicyName  --type  value  --auto-scaling-group  value  --adjustment
       value [--cooldown  value ]  [General Options]

This command outputs a ARN that will be needed in order to associate the policy with a cloudwatch alarm - note down the ARN. Each command will return a different ARN - i.e. you will have 2 ARNs - remember which one is for which policy.

For example:

Scale-up:

 as-put-scaling-policy HighMemPolicy --auto-scaling-group as-group-name  --adjustment=1 --type ChangeInCapacity  --cooldown 300

Scale-down:

as-put-scaling-policy LowMemPolicy --auto-scaling-group as-group-name  --adjustment=-1 --type ChangeInCapacity  --cooldown 300

Example return:

POLICY-ARN arn:aws:autoscaling:us-east-1:0123456789:scalingPolicy/abc-1234-def-567

Create a Cloudwatch alarm for each case: We need to use our collected data to trigger an alarm under specific circumstances, and then have that alarm execute the appropriate scaling policy. You will need 2 alarms - one for the upper threshold, and one for the lower threshold.

mon-put-metric-alarm
        AlarmName  --comparison-operator  value  --evaluation-periods  value
        --metric-name  value  --namespace  value  --period  value  --statistic
       value  --threshold  value [--actions-enabled  value ] [--alarm-actions
       value[,value...] ] [--alarm-description  value ] [--dimensions
       "key1=value1,key2=value2..." ] [--insufficient-data-actions
       value[,value...] ] [--ok-actions  value[,value...] ] [--unit  value ]
        [General Options]

For example:

Upper threshold:

mon-put-metric-alarm HighMemAlarm  --comparison-operator  GreaterThanThreshold  --evaluation-periods  4 --metric-name  UsedMemoryPercent  --namespace  "AWS/EC2"  --period  60  --statistic Average --threshold  85 --alarm-actions arn:aws:autoscaling:us-east-1:0123456789:scalingPolicy/abc-1234-def-567 --dimensions "AutoScalingGroupName=as-group-name"

Lower threshold:

mon-put-metric-alarm LowMemAlarm  --comparison-operator  LessThanThreshold --evaluation-periods  4 --metric-name  UsedMemoryPercent --namespace  "AWS/EC2"  --period  60  --statistic Average --threshold  60  --alarm-actions arn:aws:autoscaling:us-east-1:0123456789:scalingPolicy/bcd-2345-efg-678 --dimensions "AutoScalingGroupName=as-group-name"

In the above example, we are using the metric 'UsedMemoryPercent' (from the forum script) and are looking at the 'average over 60 seconds'. For the first alarm, if that average exceeds 85% for 4 consecutive samples (i.e. 4 minutes) we will trigger the alarm (and execute the action), For the second alarm, we look for 'less than 60%'.

Good references include:

Run the command with --help to see the details of the parameters.

share|improve this answer
    
I executed the as-create-or-update-trigger, I got a message from the console "DEPRECATED: This command is deprecated and included only to facilitate migration to the new trigger mechanism. You should use this command for migration purposes only. as-create-or-update-trigger: Service error: You cannot create a trigger if the group has policies attached to it. AWSRequestId:c44fd053-3d23-11e1-82c5-ed457ad103a5" –  Jeevan Dongre Jan 12 '12 at 13:48
1  
The as-create-or-update-trigger is an older command, I guess it is no longer commonly used. I have updated the answer to use policies instead. (Check what policies you have already setup with as-describe-policies, if you have conflicting policies, delete them with as-delete-policy) –  cyberx86 Jan 12 '12 at 14:36
1  
@JeevanDongre: I am not sure what you mean. The red lines should correspond to your alarm values (e.g. 85% and 60% from the examples I used above). If you can see the memory metrics (i.e. the graphs), what other 'stats' are you looking for (perhaps post a picture of what you see vs what you expect to see (e.g. from another metric))? –  cyberx86 Jan 13 '12 at 21:21
1  
A quick glance at your image suggests you are using 'FreeMemoryBytes' as your metric instead of 'UsedMemoryPercent' - using 80 and 40 bytes as your thresholds wouldn't make sense. Check that your metric names are correct. Also, increase --period to around 300 since the free metrics only have a resolution of 5 minutes (perhaps decrease --evaluation-periods to 2). Let me know if you get it working. I'll see if I can reproduce your error and get back to you in a few hours. –  cyberx86 Jan 14 '12 at 11:32
1  
Managed to reproduce it - the namespace doesn't match - the forum script used --namespace "System/Linux" while I used --namespace "AWS/EC2" - change either one so that it matches the other and everything should work. Let me know if you get it working. (Thanks for the compliment regarding my site, glad you liked it.) –  cyberx86 Jan 14 '12 at 11:47

You can make use of the template script I provided here because it's the same way I do the auto-scaling from the EC2 instance itself. I only have to add the specific commands to auto-scale right after the lines <SEND_INFORMATION_GATHERED_BY_EMAIL_HERE> because I also want to know what causes what. But make sure that you have the necessary AWS provided tools inside your EC2 instance.

Good luck!

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.