How to get the amount of free disk space with CloudWatch?

Question

Is it possible to get the amount of free disk space (df) with CloudWatch? If so, what metric represents the free disk space?

Answer 1

Update

AWS meanwhile provides their own sample scripts for monitoring memory and disk space usage on your Amazon EC2 instances running Linux at least, see New Amazon CloudWatch Monitoring Scripts:

You can run these scripts on your instances and configure them to report memory and disk space usage metrics to Amazon CloudWatch. Once the metrics are submitted to CloudWatch, you can view graphs, calculate statistics and set alarms on them in the CloudWatch console or via the CloudWatch API.

[...]

To learn more about how to use the scripts, including installation, setup and configuration, please visit "Amazon CloudWatch Monitoring Scripts for Linux" in the Amazon CloudWatch Developer Guide.

Please note though, that many popular 3rd party monitoring solutions provide respective samples or full fledged solutions for memory and disk space usage monitoring as well already, so one usually doesn't need to reinvent the wheel. The AWS provided scripts are fairly decent and extensive as well though, i.e. they cover the use case more thorough than most custom one off scripts floating around.

---

Initial Answer

No, this is not possible (yet). See the AWS staff answers on the related question cloudwatch for memory usage, which covers the reason for this as well:

[...] Right now, you do not need to deploy anything in you AMI to monitor your instance. Metrics like memory utilization and disk space require us to look into the OS running in the instance and that is why we do not have these valuable metrics.

We are looking at ways to provide more insight into your OS and applications and will have more details as we firm up the plans.

I doubt that this functionality will emerge anytime soon though, as the required cross platform agent functionality is readily available from a couple of other vendors offering dedicated monitoring solutions already, and this task is all but simple - it would definitely be a huge win to have support for this built into AWS directly of course.

Answer 2

Here's a better version of the same script that retrieves the volumes attached to an instance and reports each volume's usage.

note the substition of /dev/xvd for /dev/sd

also note that you need the ec2-api tools and cloudwatch tools installed along with a certificate to make those work

#!/bin/bash

export JAVA_HOME=
export AWS_CREDENTIAL_FILE=
export EC2_CERT=
export EC2_PRIVATE_KEY=
export EC2_HOME=
export AWS_CLOUDWATCH_HOME=

INSTANCE_ID=$(wget -q -O - http://169.254.169.254/latest/meta-data/instance-id)
VOLUME_LIST=$($EC2_HOME/bin/ec2-describe-volumes | grep ${INSTANCE_ID} | awk '{ print $2 ";" $4}')

for VOLUME_LINE in $(echo $VOLUME_LIST); do
        VOLUME_NAME=${VOLUME_LINE%;*}
        DEVICE_NAME=/dev/xvd${VOLUME_LINE#*;/dev/sd}
        USAGE=$(df $DEVICE_NAME | perl -ne 'print "$1" if /(\d+)\%/')
        $AWS_CLOUDWATCH_HOME/bin/mon-put-data -v $USAGE -d "Volume=$VOLUME_NAME" -m UsedStoragePercentage -u Percent -n "NAMESPACE" --show-request
done

Answer 3

I put together a bash script to do this using the custom cloudwatch metrics

http://aws.typepad.com/aws/2011/05/amazon-cloudwatch-user-defined-metrics.html

replace XXXX with your own values

#!/bin/sh

export JAVA_HOME=/usr/lib/jvm/java-6-sun
export AWS_CREDENTIAL_FILE=XXXX 
export AWS_CLOUDWATCH_HOME=XXXX

INSTANCE_ID=$(wget -q -O - http://169.254.169.254/latest/meta-data/instance-id)

$AWS_CLOUDWATCH_HOME/bin/mon-put-data -v $(df / | perl -ne 'print "$1" if /(\d+)\%/') -d "InstanceId=$INSTANCE_ID,Volume=root" -u Percent -m UsedStoragePercentage -n "XXXX" --show-request

put this in a cron job, and you'll have a metric you can put alarms on. I decided to go with percentages and instance specific mount points instead of monitoring the volume itself because it suited my needs. I tend to resize and switch around volumes more than I'm restarting instances. Your mileage may vary.

Answer 4

Unhappy with Amazon's sample scripts or the various 3rd party scripts that use their Perl-based command-line tools, we wrote our own script in Python: aws-cloudwatch-linux-metrics.

Single file, has no dependencies, and runs on any Python interpreter.

It reads data right from the /proc filesystem and reports disk space utilization in addition to other system metrics that aren't observable from the outside:

• CPU load average
• Network connections (TCP and UDP)
• Disk space utilization
• Memory utilization

It reports these metrics with respect to both the instance id and the image id.

Answer 5

Sometime after the question was asked, AWS provided scripts with options to report disk usage.

You can run this script in crontab to report available disk space to CloudWatch. Don't forget to include the disk path.

*/5 * * * * ~/aws-scripts-mon/mon-put-instance-data.pl --disk-space-util --disk-path=/ --from-cron

The Docs give a full list of options plus example here.