Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have tried rsync for copying VMs to remote host but the deltacopy does not seem to work. I am using Virtualbox on CentOS 6.2. My VM has 1 snapshot so I just copy the snapshot file instead of the entire base image. This works when I copy to the remote host mapped as a local drive (in which case rsync deltacopy is not used).

The problem is that when I use rsync over ssh to copy a simple text file changes are copied but when I use the same method to copy the snapshot file, changes (such as a text file created on the desktop) are not copied. Has anyone successfully used rsync to copy changes to a VM HDD without copying the whole file?

share|improve this question
add comment

1 Answer

I think rsync probably does do its delta copy for a virtual disk, but the disk is so large that it needs to read the whole thing to find the differences. With multiple files (like a normal file system, not a virtual disk), rsync can use the individual timestamps and sizes of all your files, but with a virtual disk it just has the 1 file.

So if you are copying a file locally, it is reading the disk twice on the same machine (2 files, source and destination) making it very slow still, and if it's remote, the transfer over the wire is low, but the full disk is stll read on both sides by the 2 separate rsync processes during the rolling diff process. I haven't perfectly verified this, but watching iostat (with all disk activity not just my rsync test), I found the write speed to be very low compared to the read speed.

Also, when you copy with rsync, it creates a temporary file before overwriting the destination file. To avoid this, you can use --inplace. This way you never have a second copy written, but you still have the whole disk read, so it is not a perfect solution.

Originally I said: I don't think there is a way around reading the whole file... to do this you need something other than rsync, such as a copy-on-write file system like btrfs or zfs with incremental sending ability, which is the ability to already know the differences just from the file system metadata rather than reading all the data again.

Edit: Today I realized you could probably avoid reading the whole file if the file was split up in many parts, so each part has a different timestamp, etc. with the vmdk format's Split2G variant, which splits the file up into many 2GB files (but I haven't verified that the parts have different timestamps).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.