We want to take a backup of everything on our Debian server, which is running remotely on the other side of the world (hosted by Linode), without shutting it down.
This system is running shell, email, XMPP/prosody and web, with a couple of simple nginx setups.
We want to backup files related to those things just to be safe. For example, files users have stored in their home directories.
We don't need to exactly copy the existing setup wrt every single /etc file; instead, the reason we're even doing the back up in the first place is so we can move it all to a new setup (newer version of Debian still on Linode).
I see that Linode offers a backup service. But in the long term we also need backups of our own, here, in case they go under or something else weird happens.
The reason this question exists is that when I've tried making backups in the past I've kept making either of these two mistakes:
- I've went "OK, I'll just copy
/and everything under it" and then got stuck in some weird infinite loop either because of the drive I was copying to was mounted under /media/backup and it was copying itself recursively [obv that specific problem not applicable here since we're gonna backup over rsync or similar] or it has gotten stuck trying to copy some "alive" stuff in /proc or /var or whatever, like trying to keep up with ever-changing logs, or
- I've went "OK, I'll just grab the bare minimum of what we need… hmm, everyone's home directories, and our web server directories (all under
/var) and let's snag a copy of
/etcand all the old mails under /var/vmail" and then I’ve invariably fucked up file permissions or timestamps (gonna make sure I don't backup unix files to a FAT drive this time) or forgot something ("oh, shoot, I had some custom scripts in /usr/local/bin that I never stored anywhere else, I forgot to get those, guess they're gone now").
So obv copying the entire drive straight up has lead to pitfalls and copying directories selectively has lead to pitfalls. I want to know how to do it right.
The Server Fault question What's needed for a complete backup system? covers philosophy and good practices, but I am looking for these more specific details of:
- Which directories do I need to copy and which do I exclude (given that it’s a system that’s currently running & and serving up a wiki, XMPP chat, email — with new messages coming in while the copy job is running)
- What file attributes such as time stamps & owner & group do I need to present and how do I do that? ← I think I can answer this half of the question myself with something like… um…
rsync -HXazI think is a good option for us? The
-zobv not really related to the question which is "what do I preserve"
A lot of the backup advice I see, like using
dd, seems to presuppose that the drive is unmounted and not in use. But am I not supposed to exclude "living" directories like /proc and some of the subdirectories under /var (however, some of the stuff under /var I know we definitely do need to keep) and /mount? What else is there that I need to think about in this situation? Then I guess I can just snarf it with rsync and using a bunch of
Or are there better ideas, especially FOSS friendly ones?