AWS - Script to create EC2 snapshots and automatically rename them

Question

I'm currently trying to setup a script (using AWS CLI from an Ubuntu server) that will do the following:

• Create a snapshot of every existing volume. Those volumes already have a NAME tag (Like SERVER1, SERVER1DATA, SERVER2, SERVER2DATA etc).

• Rename those snapshots taken with the same tag NAMES (so then, on the AWS console i could filter them by date and would be able to easily identificate them).

The automatic rename is my main issue here.

I've been toying with ec2-describe-volumes , ec2-create-snapshot and ec2addtag commands but my scripting is not that good. So far i've been able to setup this by creating a list with VOLUME NAME and NAME TAG (those parameters are then taken by the ec2addtag) but i would have to manually update that list every time i add or remove a volume on the environment.

Any help will be greatly appreciated it.

Answer 1

I'm not positive I understand your question completely, but if what you want is to be able to generate a list of your volumes, along with name tags, something like this might work:

aws ec2 describe-tags --query "Tags[*].{Name:Value,ResourceId:ResourceId}" --filters "Name=key,Values=Name"  --filters "Name=resource-type,Values=volume" --output json

Basically what that says is "Give me the resource id and the value of the tag "name" for every resource of type "Volume". In this case, I specified json as the output. You may also specify "text" or "table" depending on your needs.

Another approach:

aws ec2 describe-volumes --query "Volumes[*].{ID:VolumeId}" --output text

This would return a list of your volumes. If you piped this to a text file, the file would just contain a list of volume identifiers - one per line.

You could then get the Name tag for each volume in the list with something like this:

aws ec2 describe-tags --query "Tags[*].{Name:Value,ResourceId:ResourceId}" --filters "Name=key,Values=Name"  --filters "Name=resource-type,Values=volume" --filters "Name=resource-id,Values=vol-2e293949" --output json

This is basically saying "give me the resource id and the value of the tag name for the specified id vol-2e293949.

As you can see the CLI commands can be hard to read and the filtering and querying is a bit difficult. (These examples use a recent version of the AWS CLI)

Answer 2

As for extracting the snapshot ID from your create-snapshot command, you can do it without needing awk/sed/grep/etc. by using a few CLI features (query and output).

For example:

$ SNAP_ID=`aws ec2 create-snapshot --cli-input-json file://$temp_file --query 'SnapshotId' --output text`
$
$ echo "snap id: "$SNAP_ID
snap id: snap-aaaabbbb
$
$ aws ec2 create-tags --resources $SNAP_ID --tags Key=Name,Value=$SOME_NAME
$

Note that with an output of json instead of "text" it wraps the returned snapshot id with double quotes per JSON standard.

Answer 3

A while back I wrote this bash script to do exactly this for me. Basically, you install it on each EC2 instance that you want to back up, and providing you give it a private key, a certificate file and the EC2 API tools, this will self-discover the instance ID, the volumes attached to it, and create a snapshot for each volume.

You must tag the volumes with Name and Device. Name is just a user-friendly name that appears in the snapshot's description. Device is is the actual device name on the server. /dev/sdf for instance. Finally, it will rotate your snapshots once a predefined limit has been reached. The default is 50. When the 51st snapshot is created, the oldest one will be deleted.

Answer 4

#!/bin/bash

aws ec2 describe-volumes --query "Volumes[*].{ID:VolumeId}" --output text > /tmp/volumes

for i in $(cat /tmp/volumes); do

aws ec2 create-snapshot --volume-id $i | awk {'print $2'} | grep snap* | sed 's/\"//g'|sed 's/\,//g' > /tmp/snapname

SNAPENAME=$(cat /tmp/snapname)

TAGNAME=$(aws ec2 describe-tags --query "Tags[*].{Name:Value,ResourceId:ResourceId}" --filters "Name=key,Values=Name" --filters "Name=resource-type,Values=volume" --filters "Name=resource-id,Values=$i" --output text | awk '{ print $1 }') 

sleep 5

aws ec2 create-tags --resources $SNAPENAME --tags Key=Name,Value=$TAGNAME >/dev/null

done

It's a little bit messy and perhaps unnecessary long, but it does what required: Create a snapshot of all available volumes and (if those had NAME tags) then rename them with those NAME tags.