4

I am running a typical LAMP server. I need to back up the data on this machine over the network without service interruption. The backup system understands SSH, FTP, SMB, NFS & iSCSI. What would be the best approach to accomplish this?

Improve this question

12 Answers 12

Reset to default
6

You could use 'scp' (which uses SSH) to backup the data but the better option to look into is setting up 'rsync' : Replicating Webservers

rsync is pretty fast as it only mirrors the difference as opposed to doing a full copy.

Improve this answer
1
  • 1
    Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference. Especially as the linked article is now dead.
    – Mark Henderson
    May 23, 2012 at 22:47
4

Rsync is the best thing ever. However, as a general purpose backup tools it has a couple of failings (not failings in itself, its a fine file-copy tool). You have 2 issues:

  1. backup sets will be made over time, so the first file copied will be older than the last file copied. Yup, obvious, but if both files need to be in sync, then you have a backup that doesn't reflect the server state. This might not be an issue most times, but if it ever is, you will have a problem when you restore.

  2. some data doesn't copy well, eg a MySQL DB file. If the file is open, and some of its contents are in-memory, then the true state of the server will not be backed up.

The solution is reasonably simple: ensure all running services that are important flush their data to disk immediately before backup, and then ensure you take a snapshot of the disk state.

MySQL has tools to dump the databases - mysqldump. I use this to create a backup sql file that is backed up, I ignore the mysql files themselves after this, when I come to restore, I know I can restore the dumps. LVM has the facilities to take disk snapshots. These create a spare partition and all disk writes from the time the snapshot is taken transparently go to the snapshot partition, meaning your original drive remains unchanged. After you have taken the backup, delete the snapshot and all changes are 'committed' to the main disk.

Alternatively, you could use a virtualization system to hold your web server, then backups involve suspending the VM image, and copying its files (using rsync!) to a backup destination. Restoring is as simple as copying the backup files back and un-suspending them (you will have a short downtime while this happens, though some VM systems can minimise the downtime if you suspend and use a snapshot filesystem).

The best alternative, if you have money, is to use something like r1soft's continuous data protection, which constantly backs up changes.

Improve this answer
2

Rsync is great, but rdiff-backup is even better. Not only does it keep a mirror of all your files, it also allows you to restore old versions from previous backups if you want. It only saves the parts of the file that changed (the "reverse diffs"), so you're not saving a whole bunch of extra data to get the restore functionality.

It uses the same algorithms as rsync, but it's quite a bit more powerful and useful.

Also, duplicity is a system that works much like rdiff-backup, but it does everything on the client side (i.e. it doesn't need to be installed on the server). It can also encrypt the backups before you send them, and it can be configured to work with work with Amazon S3 storage.

Improve this answer
1

Rsync is the way to go for simple setups.

For bigger setups, consider BackupPC :-)

Improve this answer
1

I would go with DRBD to sync the web content and the web configs onto another server or share.

Improve this answer
1
  • DRBD is waaayyy too overkill for this setup.
    – Antoine Benkemoun
    May 28, 2009 at 13:11
1

I currently backup several unix systems using ssh keys and rsync over ssh. It works great and is fast as rsync only syncs over the differences. I dump a bunch of rsync backups to one linux box, then backup exec can run a job to send that data to tape later on. The bash scripts are simple enough to write

Improve this answer
1
  • I second this, I use a similar technique
    – devin
    May 29, 2009 at 13:42
1

Many of the commenters have noted that rsync is a fast and convenient way of making backups, but it's important to realise that the most common cause of data loss is not hardware/media failure, but human error ("oops, I just deleted the wrong bunch of files").

Having incremental backups, with point in time recovery is essential. Dirvish - www.dirvish.com - is an excellent tool built on top of rsync which allows for efficient snapshot recovery and backups with rsync.

Another important consideration is security - performing a backup generally implies giving a remote process root-level visibility of your system. With rsync + ssh, one great way of limiting the potential damage is to use SSH's command limiting. You can limit the actions a public key can execute, e.g.:

command="sudo /usr/bin/rsync -Rlptd / -",no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa AAAAB3N....

in an authorized_keys file - ensuring (together with correct sudo limiting) that a remote key is very restricted in what it is permitted to execute.

Improve this answer
0

If you go the roll your own route with rsync, then you might also try Unison. If you'd prefer a backup system then I would try either Bacula or Amanda.

Improve this answer
0

I quite like using rdiff-backup. This uses rsync under-the-hood to store revisions of files (diffs), allowing you to easily rollback a file or tree to a certain time.

Improve this answer
0

rsync is the simplest solution. Personally I use DAR to make differential backups and then rsync to transfer them to another server. Don't forget that you can't backup your MySQL database by simply backuping /var/lib/mysql ; it will be likely be in a corrupt state. Use mysqldump instead.

Improve this answer
0

You might want to take a look at rsnapshot too (www.rsnapshot.org). It is probably available as an installation package for the Linux distribution you are using.

Rsnapshot uses hard links to store backup snapshots so that several snapshots over a period of time consume disk approximately only for the total size of the oldest snapshot and the size of changed files since then. Since it is using rsync for file transfer, only changed files are transferred since last snapshot.

You can define intervals of snapshots and for how long back the snapshots are kept. A typical setting could be to keep monthly snapshots for 6 months, weekly snapshots for one month, and daily snapshots for one week.

Setting rsnapshot up nicely requires some knowledge on ssh and/or rsync (depending on your setup) too. In a "secure" internal environment you could suffice using plain rsync as the transmission medium for rsnapshot.

Improve this answer
0

Yes! Rsync is very fast, its extraordinarily versatile file copying tool.

  • List item

1.Rsync supports for copying, links, devices, owners, groups and permissions.

2.Exclude and exclude-from options similar to GNU tar

3.Does not require super-user privileges

Local: rsync [OPTION...] SRC... [DEST]

   Access via remote shell:
     Pull: rsync [OPTION...] [USER@]HOST:SRC... [DEST]
     Push: rsync [OPTION...] SRC... [USER@]HOST:DEST

   Access via rsync daemon:
     Pull: rsync [OPTION...] [USER@]HOST::SRC... [DEST]
           rsync [OPTION...] rsync://[USER@]HOST[:PORT]/SRC... [DEST]
     Push: rsync [OPTION...] SRC... [USER@]HOST::DEST
           rsync [OPTION...] SRC... rsync://[USER@]HOST[:PORT]/DEST

   Usages with just one SRC arg and no DEST arg will list the source files instead of copying.
Improve this answer