Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am in the process of implementing a backup of a WD My Book Live Duo. This device is configured with software RAID0. The backup source directory is the root of the device, and the destination is a directory inside the same filesystem (taking appropriate care to avoid recursive copy of the backup files themselves). There is a layer of encryption for backup files (source files are clear text), what requires more computation power than that the NAS is able to deliver. Because of this, CPU-consuming opperations are carried out in a server that is in the same network.

The NAS is exporting two directories through NFS. One is read-only and contains the whole filesystem (it is the backup source), and the other is read-write (it is the backup root). Configuration details are stated at the end of the post.

The backup server uses encfs as encryption layer and rsnapshot to carry out the backup itself.

The problem is: The first step of the backup, which consists in a cp -al of the last backup directory into a new directory, is taking about two hours to complete.

I have performed the following tests in the NFS server side so far:

  • The same cp -al operation takes five minutes if executed directly in the NAS
  • dd /dev/zero ~/test.tmp shows a throughput very close to 100 MiB/s
  • top shows low CPU consumption. However, load average is high (~4)
  • iperf transfer rate is 1.000 Mpbs, so the network link is performing at the expected full capacity
  • vmstat 1 shows very high wa (waiting for IO) values (~80-90%)
  • Setting MTU to 9000 (tested with ping -M do mybackup) didn't seem to have any effect
  • Exporting with async didn't seem to have any effect

The following tests have been carried out in the NFS client side:

  • dd /dev/zero /e/test.tmp (encrypted) shows the same throughput as dd /dev/zero /b (unencrypted). So the encryption layer doesn't seem to be the problem. The transfer rate is ~30 MiB/s, which is quite low, tough. Using SMB to perform this test yields simmilar results
  • top shows low CPU consumption. load average is below 2 (the server has 2 cores)
  • vmstat 1 shows that the system is almost idle
  • Setting tcp,hard,intr,nfsvers=3,rsize=32768,wsize=32768 mount options didn't seem to have any effect

Configuration files

NFS Server

# /etc/exports

# 'mybackup' is the name of the backup server
/               mybackup(ro,sync,no_subtree_check,no_root_squash,crossmnt)

# This export has to be explicitly declared to enable clients to access /DataVolume/b
/DataVolume     mybackup(ro,async,no_subtree_check,no_root_squash)

# This is the backup root.
/DataVolume/b   mybackup(rw,async,no_subtree_check,no_root_squash)

NFS Client

# /etc/fstab

# 'ewnas' is the name of the NAS
ewnas:/                 /r      nfs     auto,noatime,tcp,hard,intr,nfsvers=3,rsize=32768,wsize=32768    0       0
ewnas:/DataVolume/b     /b      nfs     auto,noatime,tcp,hard,intr,nfsvers=3,rsize=32768,wsize=32768    0       0

I've been stuck with this for several days now. Any contribution will be very appreciated.

Thanks in advance,

Guillermo

share|improve this question

1 Answer 1

up vote 0 down vote accepted

Have you tried rsync to see if performance is better?

rsync -Paz --exclude-from 'rsync-exclude.txt' fromssh@server:path tossh@server:path

Reference discussion

share|improve this answer
    
Close. I changed my rsnapshot configuration to make it use its native cp implementation instead of system cp command (this is, I removed cmd_cp and cmd_du options). The backup process took only 10 minutes! I'm accepting your answer as a solution for giving the idea of using a different implemntation of cp. BTW, in rsync, source and destination parameters cannot both be remote... –  Guillermo López Alejos Jul 27 at 12:07
    
Thanks for the pointer on the rsync. Another option I found in some research was netcat. Glad you were able to sort it out though and thanks for the points! –  bcj Jul 28 at 12:37

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.