0

A bit of background... We run Redis on several EC2 machines running the Amazon Linux distro (seems to be based on Centos to me) and currently utilize both the RDB and AOF backups. These files are stored on EBS volumes, but we also back the files up to S3 on an hourly basis in case something happens to EBS.

Currently we copy the rdb/aof files into a new location and then execute aws s3 cp --recursive /mnt/storage/redis/${DIR} s3://backup_bucket and then clean up all the temp files.

I'd like to modify this to avoid the disk IO of copying the files around. What happens if while aws s3 is copying the old RDB file, a newer one gets created and swapped in? Or for the AOF file, which has new stuff appended to it every second, what happens to it?

Stuff that might matter: EXT4 volumes, will likely be setting up enhanceIO to pair the EBS volume with an ephemeral SSD in a writethrough mode.

0

Is there a reason why you don't use EBS snapshots? You can use those to save the whole EBS device (incremental) with a simple API call and the snapshots are saved within S3. If you need an old version back just create a volume from this snapshot and connect it to your instance instead of the broken EBS.

  • mostly because there's no mechanism for automatically cleaning up old snapshots. we use the lifecycle rules to auto-purge old backups from our S3 bucket and would love to use EBS snapshots if such a thing existed there. – Ty W Jul 22 '15 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.