15
votes

What backup solutions you use for linux server in production environment? Do you prefer open source or commercial products?

What you consider to be essential features in backup software?

Which products you do not recommend? Are there any common pitfalls in backup software selection?

1

14 Answers 14

12
votes

I use the openSource bacula. It is excellent - server/client model that works on windows or linux. There is good online support, and an active developer community.

A bit tricky to set up, but has all the features you could ever want.

There is also a nice web gui called bweb that can be used for day-to-day operation of it.

2
  • Hi, I'm trying deploy bacula at my office and its very tricky. If you have any installation documentation can you please share it.. May 18, 2010 at 11:39
  • Sorry, I just used the online documentation.
    – Brent
    May 21, 2010 at 17:42
7
votes

As far as essential features go, you need to use the right tool for the job.

If your data set is small and fits on a single volume, use rsnapshot or rdiff-backup. Both offer incremental backup, are very space efficient, and are really easy to use.

For larger or more sophisticated backups that have an offline component I use Bacula, it takes a while to setup properly, but it's rock solid and has quite a few features. I recommend coupling it with a web interface like webacula.

One good idea btw, for every server I have I use rdiff-backup to keep a daily incremental backup of /etc in /var/backups/, it costs you nothing and will save you a lot of headaches if you can't figure out why that last change to a config file is causing so much havoc.

4
votes

I use rsync, both over the network from my colo to my home, and from my home box to a removable USB drive that I swap out for one I keep in my desk at work. The script looks kind of like this:

#!/bin/sh
STARTTIME=$(date +%s)
HOUR=$(date +%H)
DOW=$(date +%a)
WEEKNUM=$(($(date +%W|sed 's/^0\?//') % 4))
LOG=/tmp/last_hour.log
echo "" > $LOG

for DEST in /media/usb[0-9] ; do
  if [ -d $DEST/allhats2 ] ; then
    echo backing up to $DEST >> $LOG
    YESTERDAY=`cat $DEST/yesterday`
    LASTHOUR=`cat $DEST/last_hour`
    PREV=$DEST/allhats2/hour$LASTHOUR
    if [ ! -d $PREV ] ; then
      echo could not find a directory at $PREV >> $LOG
      PREV=$DEST/allhats2/$YESTERDAY
      if [ ! -d $PREV ] ; then
        echo could not find a directory at $PREV >> $LOG
        PREV=$DEST/allhats2/Sat/
      fi
    fi
    if [ $HOUR = "00" ] ; then
      if [ $DOW = "Mon" ] ; then
        echo moving last monday to week$WEEKNUM
        rm -rf $DEST/allhats2/week$WEEKNUM
        mv $DEST/allhats2/Mon $DEST/allhats2/week$WEEKNUM
      fi
      echo moving last midnight to $YESTERDAY
      rm -rf $DEST/allhats2/$YESTERDAY
      mv $DEST/allhats2/hour$HOUR $DEST/allhats2/$YESTERDAY
      echo $DOW > $DEST/yesterday
    fi
    echo about to backup allhats2 to  hour $HOUR >> $LOG
    rm -rf $DEST/allhats2/hour$HOUR/
    rsync -aSuvrx --delete / /boot /home /usr /var /backup_2/dbs --link-dest=$PREV/ $DEST/allhats2/hour$HOUR/ >> $LOG
    echo $HOUR > $DEST/last_hour
  fi
done

YESTERDAY=`cat /root/yesterday`
if [ $HOUR = "01" ] ; then
  # Backup xen1
  echo about to backup xen1 to /1u_backup/xen1/$DOW/
  rm -rf /1u_backup/xen1/$DOW/
  rsync -aSuvrx --delete -e ssh --exclude /var/spool/news/ root@xen1:/ --link-dest=/1u_backup/xen1/$YESTERDAY/ /1u_backup/xen1/$DOW/

  for DEST in /media/usb[0-9] ; do
          if [ -d $DEST/xen1 ] ; then
                  echo "backing up the backup"
                  rm -rf $DEST/xen1/$DOW/
                  rsync -aSuvrx --delete /1u_backup/xen1/$DOW/ --link-dest=$DEST/xen1/$YESTERDAY/ $DEST/xen1/$DOW/ 
          fi
  done

  # Backup xen
  echo about to backup xen to /1u_backup/xen/$DOW/
  rm -rf /1u_backup/xen/$DOW/
  rsync -aSuvrx --delete -e ssh root@xen:/ --link-dest=/1u_backup/xen/$YESTERDAY/ /1u_backup/xen/$DOW/

  for DEST in /media/usb[0-9] ; do
          if [ -d $DEST/xen ] ; then
                  echo "backing up the backup"
                  rm -rf $DEST/xen/$DOW/
                  rsync -aSuvrx --delete /1u_backup/xen/$DOW/ --link-dest=$DEST/xen/$YESTERDAY/ $DEST/xen/$DOW/ 
          fi
  done

  echo done

  echo $DOW > /root/yesterday
fi
4
  • +1: ...for rsync; I use the same thing :-)
    – Jon Cage
    May 28, 2009 at 14:41
  • LuckyBackup, based on rsync. Jul 17, 2013 at 11:31
  • @NikosAlexandris, I wouldn't use a backup product with the word "lucky" in the name - I don't like the connotation that "if you're lucky, you'll be able to restore it". Jul 17, 2013 at 12:40
  • :D -- Check the "awards" in the project's homepage though. Jul 17, 2013 at 13:48
3
votes

I use jungledisk with the Rackspace cloud file storage. It just goes every night and does its thing, I just tell it what to back up, and it even keeps revisions (sort of). Offsite, and fairly inexpensive.

1
  • I too love Jungledisk. It's just easy.
    – Corey S.
    Feb 9, 2011 at 2:29
2
votes

I'm using backuppc for all my hosts. Only the 500GB maildir store was problematic because rsync really used too much RAM on the receiving side. I switched to using tar and time-stamp base incrementals for that host.

2
votes

BackupExec, with the free RALUS *NIX agent. Exports easily, doesn't require SAMBA, and fits in with the Windows servers that are also backed up by agents.

1
  • Is the Unix agent still free? The last time I looked at it there was a licensing cost associated with it - which surprised me, as it was free from at least BackupExec 10 and earlier. May 25, 2009 at 5:52
2
votes

If you're looking for a disk based system (rather than tape or offsite or somethign), I would suggest looking at BackupPC. We have had good experience with it. It seems to do a good job of pooling files to get good use out of the disk space, it's very configurable and has been rock solid for us.

2
votes

I'd suggest Bacula I currently use it in our shop. It also works with windows, if you ever find you have a need for it. :)

Another suggestion might be Amanda, however I've never used it, although the two are often compared against one another.

Bacula is a set of computer programs that permit you (or the system administrator) to manage backup, recovery, and verification of computer data across a network of computers of different kinds. In technical terms, it is a network Client/Server based backup program. Bacula is relatively easy to use and efficient, while offering many advanced storage management features that make it easy to find and recover lost or damaged files. Due to its modular design, Bacula is scalable from small single computer systems to systems consisting of hundreds of computers located over a large network.

2
votes

This document explains in a very good and detailed way how to use rsync for incremental backups on Linux.

1
vote

In the proprietary world, R1Soft is doing a very good job. Try the demo tools that they release gratis.

In the free world, I use lvm snapshots, dd and netcat to make quick backups. Just make sure dm_snapshot exists in your initrd :)

1
vote

i'm happy user of backupninja wrapper bash script. it's available in debian as package in standard repository.

depending on type of data i use directly rdiff-backup mentioned by Andrew Cholakian, or first take snapshot of LVM volume [ mentioned by tinkertim ] and then run rdiff-backup on it.

rdiff-backup does not work very efficiently over bad network links, in such a cases i run rdiff-backup locally and then use rsync to synchronize backup repository with remote server.

1
vote

Where would you like to backup to? rsync can be an invaluable tool for keeping a copy of directories in sync if you would like to backup to another linux server. You can keep time based backups by sourcing a previous backup. You are essentially getting a full backup every time but it will only pull files that have changed. Here is an example bash script that will use rsync, don't forget to read the comments.

#!/bin/bash

# If you want to automate this script you will need to generate public/private
# key pairs for the user executing this script on the remote server. 

# Change these variable to reflect where you want the backups to be stored
# and what servers will get backed up. Servers are seperated by spaces, do
# not use commas.
BACKUPLOCATION='/backups'
SERVERS=( testserver.example.com )
RSYNC='/usr/bin/rsync'

# These are variables used internally to the script DO NOT CHANGE!
TODAYSDATE=`date +%Y-%m-%d-%H`
FOLDERNAME='Backup-'$TODAYSDATE
CURRENTBACKUP='CurrentBackup'
USELINK=0

backup() {
    # Test and make sure that the folder we are backing up to is writable
    if [ -w $BACKUPLOCATION ]; then
        # The folder we are backing up to is writable no problems
        echo The backup destination is writable, continuing.
    else
        mkdir -p $BACKUPLOCATION
        if [ $? -eq 0 ]; then
            echo "Backup directory '$BACKUPLOCATION' was created"
        else
            # We can't write to the folder alert the user and stop the backup
            echo The backup destination is not writable! Backup Failed!
            exit 1
        fi
    fi

    # If there are older backups then use them as a source directory
    if [ -h "$BACKUPLOCATION/$CURRENTBACKUP" ]; then
        echo
        echo "Previous backup found... Will link to unchanged files..."
        LASTBACKUP=`ls -l $BACKUPLOCATION | grep $CURRENTBACKUP | awk '{ print $11 }'`
        echo $LASTBACKUP
        echo "Deleting Old Link..."
        rm -f $BACKUPLOCATION/$CURRENTBACKUP
        USELINK=1
    fi

    for SERVER in ${SERVERS[@]}
    do
        echo
        THISRUNLOC=$BACKUPLOCATION/$FOLDERNAME/$SERVER
        mkdir -p $THISRUNLOC

        if [ $USELINK -eq 1 ]; then
            OPT="-a --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/tmp --delete --link-dest=$LASTBACKUP/$SERVER $SERVER:/ $THISRUNLOC"
        else
            OPT="-a --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/tmp $SERVER:/ $THISRUNLOC"
        fi
        #echo "Performing: rsync $OPT"
        echo "Backing up server: $SERVER"
        $RSYNC $OPT
        if [ $? -eq 0 ]; then
            echo "Success!"
        else
            echo "Backup failed with an error code of $?"
        fi
        echo
    done

    echo "Creating New Link..."
    ln -s $BACKUPLOCATION/$FOLDERNAME $BACKUPLOCATION/$CURRENTBACKUP
    echo "Backup Complete!"
}

restore() {
        # I didn't write the restore code you can simply copy the files back
}

case $1 in
    backup)
        echo "Running the backup..."
        backup
        ;;
    restore)
        echo "Restoring backup..."
        restore
        ;;
    *)
        echo "Run this command with either backup or restore"
        ;;
esac
0
votes

Dirvish

What you consider to be essential features in backup software?

  • Being able to restore very easily
  • Doesn't waste storage or bandwidth for the backups
0
votes

I can't believe no one has mentioned Duplicity. I can't recommend it highly enough. It has multiple backends - from FTP to ssh to S3 - and does encryption (gpg) and incremental backups, and the restore process is frightfully easy.

Not the answer you're looking for? Browse other questions tagged .