Tarball backup causes massive server lag

Question

I am running three minecraft servers, 40-60 online users in average spread over the daemons, on a debian 6.0 squeeze root server.

I'm trying to configure a cronjob backup and wrote a small shell script to just compressing a single (at the moment 7.1 gb of size) directory to a specific directory.

tar -cvzf /home/backups/mcsrv1/mcsrv1-full-$DATE.tar.gz /home/mcsrv1/

nothing special.

But, through the structure of some plugins installed in each mc-server there are thousands and thousand of small files and empty directories like:

/home/mcsrv1/plugins/somepluginname/1/2/1-7/18/2r.001.mca

Now, when I'am starting the script, it takes almost 45 to 60 minutes to backup the files and directory structure and it causes MASSIVE server lags. It is almost unplayable at this time.

Even if I just want to look how big the directory is with

du -sh /home/mcsrv1

the server almost (if not) crashes.

Server specifications:

i7 2600, 32GB RAM, 3TB SATA 6GB/s, root server (physical machine)

Am i doing something wrong with the script or did i configure the server wrongly?

Looking with the command "top" when the script is running don't show any suspicios entries.

Any advice could help.

I am trying to run the script when the server is empty at 5 am. but the servers is never really empty, though.

I don't want to shut the mc-servers down in order to do a backup.

Additional

The tarball command is just the main part of the backup, the script itself is a little bit more. E.g. it displays a warning to all players and turns off automatic saving operations during the backup process.

Answer 1

You could use a wrapper script that calls the backup script and sets the CPU core the backup process will use (if you have a multi core CPU!). It has a slight benefit (not massive), but I have this low end servers.

For example, I have a web server with dual 8 core CPUs, so it has 16 cores essentially. 90% of programs including the main ones (Apache, MySQL, memcached etc) run on cores 0 & 1 (the first two on the first CPU) and 8 & 9 (the first two cores on the second CPU). So I set the backup process to run on core 15 and let the kernel scheduler "do it's thing".

To do this I use a wrapper script that is called like this;

 #!/bin/bash
 taskset -c 15 /path/to/backup.sh

Also set a nice value an ionice value for your backup process (I expect the lag is probably comming more from the IO operations rather than the CPU operations).

In your backup script try the following for ionice;

#!/bin/bash
ionice -c3 -p$$
nice -n 12 tar -cvzf /home/.....