How to schedule automatic (daily) snapshots of AWS EC2 Windows Instance?

Question

I have some Windows servers hosted on Amazon EC2. Some run Windows Server 2003 and other run Windows Server 2008. These are EBS-backed instances. Most of the instances also have some additional EBS-volumes attached.

We want to schedule a daily snapshot of the windows machines (and also the attached EBS-volumes) to S3 so that we have daily backups available.

One would think that this is a very common requirement and would be made available via the AWS Management Console, but alas, it is not. What approaches are available? How do I schedule daily snapshots on our Windows Servers?

There are several scripting examples available online for Linux, but not so much for windows. I have had a look at http://sehmer.blogspot.com/2011/04/amazon-ec2-daily-snapshot-script-for.html as well as https://github.com/ronmichael/aws-snapshot-scheduler. Has anyone used one of these approaches and does it work?

I have also considered a service like Skeddly which seems inexpensive at first glance but when you look at using it for several servers the price soon escalates to such a point where it seems a better option to create your own solution as you can then apply it to new servers in the future. With Skeddly we'll pay for each server.

How do we schedule daily snapshots of our windows instances?

Answer 1

Amazon Web Services recently announced PowerShell command line tools for Windows and it's packaged along with their AWS Tools for .NET SDK.

The AWS Powershell tools make it quite easy to create a snapshot:

New-EC2Snapshot "vol-371acd04" -Description "My Snapshot"

And you can query your snapshots like this:

PS C:\Program Files (x86)\AWS Tools\PowerShell> Get-EC2Snapshot | more


SnapshotId  : snap-18be2b28
VolumeId    : vol-371acd04
Status      : completed
StartTime   : 2012-12-28T08:17:00.000Z
Progress    : 100%
OwnerId     : 383816850479
VolumeSize  : 30
Description : My Snapshot
OwnerAlias  :
Tag         : {}

Make sure you have the AWS Powershell tools installed and just create a scheduled task that uses a powershell script similar to the snippet above to schedule your snapshots and you should be good.

Updated to query for attached EBS volumes:

To query for EBS volumes attached to your instance and then snapshot each of them you could do something like this:

# Find my instance ID from the EC2 metadata
$myInstanceID = (New-Object System.Net.WebClient).DownloadString("http://169.254.169.254/latest/meta-data/instance-id")

# Query for volumes that are attached to my Instance Id
$volumes = (Get-EC2Volume).Attachment | where {$_.InstanceId -eq $myInstanceID } | Select VolumeId

# Iterate through these volumes and snapshot each of them
foreach ($volume in $volumes)
{
    New-EC2Snapshot $volume.VolumeId -Description "My Snapshot"
}

Answer 2

2018 Update As of late 2018 there are two additional ways to automate EBS snapshots. I still use the original method of CloudWatch events as it's worked fine for years and I see no point in changing.

Ops Automator (OA)

Ops Automator is a very flexible set of lambda scripts provided by AWS. It's deployed with a CloudWatch template which is on the page above.

It's setup steps are

1. The AWS CloudFormation template launches the core framework, which includes a suite of microservices (AWS Lambda functions) that manage triggering events, resource selection, task execution, concurrency control, and completion.
2. Task configuration data, which defines the triggering event, how the task should be performed, which resources will be selected by the actions, and where these resources are located, is stored in an Amazon DynamoDB table.
3. Solution-generated AWS CloudFormation templates configure tasks based on parameters you define, and the roles necessary to perform actions across accounts.
4. The solution tracks all steps in the process, the selected resources, and the results of the actions, including possible errors, in a DynamoDB table.
5. The solution also leverages Amazon CloudWatch Logs for logging. Warning and error messages are published to a solution-created Amazon Simple Notification Service (Amazon SNS) topic which sends messages to a subscribed email address.

Data Lifecycle Manager (DLM)

DLM Documentation. This is a simpler but less flexible solution which can back up volumes every 12 or 24 hours. I'm puzzled why AWS put this limitation on DLM - weekly, monthly, or a variable frequency would've been easy to implement.

DLM is integrated into the AWS console. I'm not going to copy and paste the documentation as AWS keeps things updated well, and links rarely break.

As of 2019 DLM lets you specify shorter intervals down to two hours, but still doesn't let you specify intervals of more than 24 hours.

2017 Update

As of 2017 there is another way to create regular snapshots - using Cloudwatch Events.

This lets you schedule snapshots, but it doesn't solve the problem of the volume being in use, so it's only a partial solution. There may be a way using CloudWatch Events to trigger something that does quiesce the volume.

1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

2. In the navigation pane, choose Events.

3. Choose Create rule.

4. For Event Source, do the following:

   -> Choose Schedule.

   -> Choose Fixed rate of and specify the schedule interval (for example, 5 minutes). Alternatively, choose Cron expression and specify a Cron expression (for example, every 15 minutes Monday through Friday, starting at the current time).

5. For Targets, choose Add target and then select EC2 Create Snapshot API call.

6. For Volume ID, choose an EBS volume.

7. Choose Configure details.

8. For Rule definition, type a name and description for the rule.

9. For AWS permissions, choose the option to create a new role. This opens the IAM console in a new tab. The new role grants the built-in target permission to access resources on your behalf. Choose Allow. The tab with the IAM window closes.

10. Choose Create rule.

Answer 3

Jumping in on an old answer that started me off on my search. Kudos to Ameer Deen above for the scripts, btw.

Found this post which dives in much deeper to PowerShell scripts to fully automate the data-gathering and EBS snapshot process:

http://messor.com/aws-disaster-recovery-automation-w-powershell/

Then of course discovered that those were all written for the now-depreciated v1.0 of the AWS SDK, so I spent the last day fixing that and getting them all working with v2.0:

https://github.com/noahlh/aws-automated-backup-powershell

Enjoy!

Answer 4

AutomatiCloud does exactly what you need. It is an easy to use windows tool where you can schedule backups for your EBS Volumes or RDS instances: www.automaticoud.net

You can

• create snapshots and AMIs based on instance-IDs or tags
• create dynamic tags using macros
• define 3 levels of retention periods (GFS-backup)
• send email notifications
• add VSS agent for windows instances
• run scripts before/after backup
• and much more...

Answer 5

You can now use AWS Lambda to create AMIs automatically. The whole setup should be completed in around 10 minutes along with the schedule that you like. Look at their Github repo to contribute also. Run the script by yourself and avoid giving permissions to third parties.

Answer 6

You can write a script that talks to the EC2 API to schedule those snapshots. It can be run on one of your existing instances or you could set up a small linux instance for this purpose on a t1.micro whose sole function is to kick off the snapshots for the Windows hosts. The script could be put on the linux host as a cron job to run daily at the time you choose.

The first link is an example of scripting that from a Windows host and it could easily be automated with Scheduled Tasks in Windows.

I have not used the application you linked on your second link.

Answer 7

The current EBS Snapshot capabilities for Windows Instances are explained here: http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ebs-creating-snapshot.html

In order to get a consistent snapshot, you have to pause writes to the volume long enough for the snapshot to complete. Typically, this means you that you have to unmount the volume. The AWS doc does not discuss how to do this via commandline on Windows, but you can find out about that in this superuser thread: https://superuser.com/questions/704870/mount-and-dismount-hard-drive-through-a-script-software

To get a snapshot of the system boot volume, you need to stop your instance.

Hope that helps.

Answer 8

I realize this is an old question, but I'll share our solution.

We use a service called Ylastic. Their Pro plan ($50/mo) includes scheduling of EBS snapshots, not limiting the number of servers or disks. It will also remove older backups if you want. You can create multiple schedules (e.g. daily, weekly, etc.) each with their own retention policies.

The interface also lets you create new instances from backups.

It's worked reliably for us for years, with about twenty instances and forty EBS volumes. There are probably cheaper methods but the price is just low enough that we haven't bothered building and maintaining something ourselves.

See http://blog.ylastic.com/post/148519929496/multi-region-ebs-instance-backup-management for details on this feature.