I currently have a Linux server running with around 10,000 users daily on it. The hosting provider offers a backup which I'm also using. Although I trust my hoster, I would like to have an offsite backup, just in case the host goes down for a longer time or goes bankrupt (you never know). My idea was to tar
and split the data and copy the archive to my Amazon S3 account, but I'm wondering if that's the best idea.
-
After some testing with tarsnap and jungledisk, I settled for another cheap server in another datacenter. As a backup tool, I'm using rsnapshot since the backup speed (after the initial upload) is very fast and so is the recovery. tarsnap was too slow for recovery and the jungledisk client was too flaky.– andypaAug 7, 2012 at 21:25
4 Answers
Best was to offsite-backup 30GB of data?
I'd probably say rsync with your Amazon S3, but keep in mind that its 30gb, so bandwidth costs will be high (if you pay 95%) and it'll take a long time to get your initial push up there. Once your data is up there just keep the files rsynced nightly / weekly / whatever your preference is.
however, Thats ONLY the backup side, dont forget about recovery. I'd purchase a second server in a second datacenter, have similar builds, and rsync the changes nightly.
-
My final solution: github.com/pcorliss/s3cmd-modification <-- the --parallel modification, which enables multiple uploads plus S3 versioning is the best solution so far for me.– andypaOct 7, 2012 at 8:52
An S3 account isn't impractical for that size of data set -- at 10Mbps, it'll take about 9 hours to transfer all that data. Whatever you do, though, think about recovery -- are you OK with being down for the 9 hours (on top of whatever provisioning time you need for your new machine) it takes to download your 30GB of data back onto your server? Keep an eye on your data set growth, too, and keep re-evaluating that choice. At some point, it may make sense to find someone who's willing to ship you physical media (a bunch of DVDs or a USB hard drive), but for now a purely network-accessable system should suit you fine.
-
The server has a 100Mbps connection and during the night, the network connection isn't heavily used. If I'm not ok with a downtime, the only other thing would be to have a parallel server running at a different hoster, correct?– andypaJan 20, 2010 at 6:09
-
2Downtime avoidance, in general, is a large and complicated issue, not something to be covered in a comment on an unrelated answer.– womble ♦Jan 20, 2010 at 6:28
-
1
You may also want to look into JungleDisk as well. There service basically adds decent encryption and interface/app to S3 allowing for realtime data backup.