1

I'm currently in the process of "designing" my personal (Linux-based) IT, specifically backups.

I've got a few servers which generate data worth backing up (databases, files, etc.) and would like to put these in a central backup location (which is again backed up somewhere else).

Currently I'm having thoughts on security: What happens if a machine gets owned?

Originally I planned to have all the systems use the same backup user, but this turns out to be a problem: If one of the clients gets owned, he can change arbitrary other backup data.

I'm considering creating one backup user per data source I have. Since the number of data sources I have is fairly constant, this isn't a problem for me. However I feel like I am violating the DRY principle: Surely someone somewhere already thought of this problem, are there any enterprise/well-known tools I've overseen? Or is there an entirely different way I could implement this?

  • Can you use file permissions to make your repository write only, no delete or replace? If you use incremental backups that may be sufficient, though things like index files may make it difficult. If you're not doing incremental backups I would call this a copy, rather than a backup. – Tim Feb 11 '19 at 20:47
  • @Tim Interesting thought. I wouldn't know how to do this if all services share the backup user - anything added (e.g. chattr +i) can be removed later. Maybe chowning them all to root (or a second user) works, but that feels like a hack to me. – ThreeFx Feb 11 '19 at 20:52
1

Solve each of the realistic threats with a tool that is good at the one thing it does.

  1. Owned backup target compromises all source backups

    • encrypt & sign incremental backups (e.g duplicity + gnupg2)
    • backup target should never store any decryption keys for the data it receives
  2. Whoever owns one source might want to compromise other sources backups

    • user separation (e.g. one chroot user per source)
  3. Using backup upload credentials, attacker compromises that machines older backups

    • create & restore from snapshots on the backup target (e.g. lvm2, zfs)
  4. Compromised source attempts denial of service, preventing other sources backups

    • disk space limits per machine (e.g. quota)
    • credentials for transfer method only (e.g. openssh ForceCommand internal-sftp)
  5. Compromised source attempts to cause admin mistake during restore or backup-backup

    • verification procedure including signature checks (gnupg2 is only one step)
    • automation to reduce margin for human error
  6. Backups might get compromised between source and initial backup target

    • transfer through mutually authenticated tunnel (e.g. openssh)
1

Borg backup provides per client repositories using a single user on the central repo.

You should encrypt all your backups at rest, either OS/hardware level and/or repository level encryption.

https://borgbackup.readthedocs.io/en/stable/deployment/central-backup-server.html#restrictions

  • I'm stopping using Borg, as it creates a new large file every night when the backup size isn't really large. That means additional unnecessary network usage. I'm starting to use Restic instead. Restic can store backups into a few places, and has a lot of great features. One downside is it doesn't do compression, though it does de-duplication. – Tim Feb 11 '19 at 21:53
1

I worked at a place that did something similar to what you're looking at.

We had a variety of servers - mostly Linux - and they would all back themselves up to a central server, mostly using NFS. Our CRM system was developed in-house so it could be rebuilt if necessary, and it would pause itself every hour and back up its databases. The main file server did a backup every night, either full or incremental.

The central backup server ran Crashplan for an offsite backup on a continuous basis, and we'd archive off about one full backup per year to another drive.

Any one server getting compromised could only affect that server's backups, and not the archives or the offsite. If the backup server was compromised, it could affect all the "active" backups, but not the archives and not the offsite copies. We weren't worried about the server being compromised, but if there something happened to the drives where we stored the backups, we figured it was good enough to be able to go to the offisite copy.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.