Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

If I am running a modern vmware ESXi system, I can drop in a statically linked rsync binary and rsync files to any destination over SSH.

I'm trying to understand why most (all ?) backup of vmware guests is not done this way.

If the VM is running, you can simply use 'vim-cmd vmsvc/snapshot.create' to create a snapshot and then rsync that snapshot to the remote host. (there's even an option to "quiesce" the snapshot)

OR, if you want a more robust backup, you can gracefully halt the VM and rsync over the vmdk file(s).

So ... it seems like I am a simple shell script away from all the backups I ever wanted to do, simply and easily, using plain old rsync.

What am I missing here ?

share|improve this question
1  
Because if a single file changes in the VM you will have to backup the whole vmdk? –  faker Jun 25 at 22:14
    
No, rsync will update a single file efficiently with just the changes since the last transfer. Certainly the operations of the VM could produce a LOT more changes than you expect it to, but it's not going to make you resend the entire vmdk ... –  user227963 Jun 25 at 22:47
    
Other than the fact you shouldn't use the esxi shell for anything other than maintenance, the esxi OS is not made to work in that way, and you would be unsupported, I think you are misunderstanding the concept of a snapshot. The snapshot in this case is a delta. So if you take a snap and copy it straight away, it would be tiny, and contain almost no information. You are thinking of a backend storage snapshot, and yes people back up VMs in this way –  Rqomey Jun 27 at 12:08
1  
@Rqomey - there are different kind of "snapshots" in ESXi. You're talking about the one kind that is visible via the vSphere Client - but using the API you got other options, e.g.: full clone. –  masi Jun 27 at 15:59
    
@MASI Do you mean a clone then as opposed to a snapshot? ;) –  Rqomey Jun 30 at 10:27

3 Answers 3

  • Because the transfer speeds out of the ESXi console are purposefully limited.
  • Because this isn't scalable in any way.
  • Because you'd have to drop a statically-compiled rsync binary onto the ESXi host.
  • Because the VMs, the VMDKs, their ramdisk files and other components can change enough to make rsync a losing proposition... do you really want to re-sync a 200GB VM that was rebooted and had a small number of files change?
  • Because of CPU/memory resource requirements on the source or destination. Rsync isn't free.
  • Because there are other products on the market, both third-party and VMware-provided. Look up Changed Block Tracking.
  • Because ESXi is NOT a general-purpose operating system.

Also see: Install rsync on VMware ESX 4.1 server

share|improve this answer
1  
Outstanding answer. –  EEAA Jun 25 at 22:24
3  
They aren't... I mean, it's in the name: ghettoVCB. There are better solutions out there. Veeam, vSphere Data Protection, etc. –  ewwhite Jun 25 at 22:34
2  
You could certainly use the rsync method if you switch over to xen/kvm. –  Zoredache Jun 25 at 23:01
8  
@user227963 Rsync is also rather inefficient on both - large number of files as well as large files. And while it might not have to resend the entire file over the wire, it will have to reread it on source and destination alike. CBT will help you here, but rsync knows nothing about CBT. –  the-wabbit Jun 25 at 23:04
1  
@user227963 copying files is simple. Now make it fast and not a resource hog on large files with small constant changes. rsync is decent but nowhere near the performance of anything with insider info on which blocks changed. –  JamesRyan Jun 26 at 9:29

I used to do just this a few years back. (edit: with VMWare running on CentOS hosts, not ESXi admittedly)

Every night I had a script that would suspend a VM, rsync the files from disk to the backup server and then start the VMs again. It worked quite well except...

Rsync doesn't work very well with a 2GB file.

Its not because rsync isn't brilliant, it more that each 2GB vmdk file changes in ways that are very opaque to rsync, even small changes to the enclosed filesystem produce changes in the vmdk (or all vmdks for some reason) which I blamed on Windows, either automatically defragging or otherwise doing all the other things it does that don't matter if you're running a real system, but show up when you are trying to rsync a VM!

I think the rsync mechanism for detecting changes don't work very well on a 2GB file, whilst it quite often skipped chunks of the start of the vmdk, once it started to find a difference it would simply copy the rest of the file. I don't know if that's an issue with rsync not being able to detect a moved chunk of binary data, or with a lack of memory on the source box, or whether the vmdk just got updated all the way through. It doesn't matter as the result wasd the same - majority of the vmdk got copied.

In the end I simply copied any changed files and overwrite them, still using rsync. I also had better performance simply overwriting the backup file instead of letting rsync copy and replace what was there.

Our backup server wasn't the fastest either and it got to the point where overnight wasn't long enough to back up all running VMs.

However, when we did need to restore a VM, it was really easy and worked beautifully.

share|improve this answer
    
Ok, that's very helpful. I know a bit about how rsync works, and I can tell you it has nothing to do with the size of the file - but what you are describing is that much more of the file changes than you expect it to ... that is to say, you run the VM for a day, and you only do a few little things with it, and then you stop it ... but the vmdk file changed by 30-40% (even though you did very little). So rsync would do just fine, it just has a lot of work to do ... more than you expected. Thanks! –  user227963 Jun 26 at 16:11
1  
But then ... the question this raises ... how do the "professional" tools do it ? What kind of magic are they doing that is somehow more optimal than what rsync (or scp, or even cp) would do ? At the end of the day, you have a unix environment (the ESXi console) and you want to move a file in or out of it ... what secrets could there be involved with that ? –  user227963 Jun 26 at 16:14
    
@user227963 The professional tools leverage features like the changed-block-tracking or have access to other vSphere or ESXi APIs. –  ewwhite Jun 27 at 12:44

Rsyncing a single file is not a backup solution,

what do you do when something happened to the vm and files were deleted, but you only noticed this after your rsync has run again? You will have overwritten the good 'backup' of your files with the bad image now.

If you want backup you need to keep the old versions somewhere, or the diff's. Rsync will only copy over the diffs for you, but it will not store only the diff's, but overwrite the previous file.

There might be options for you here, with rsync, and a copy-on-write filesystem with versioning information, which will in effect store the diffs every time your rsync script runs. This solutions starts to get a bit more complicated already, so this is why people resort to known working solutions imho.

share|improve this answer
    
There is certainly a lot more complexity involved here than I originally thought, but what you're mentioning is not a problem. Certainly if you blindly ran rsync over and over you would run into trouble, as you suggest, but there are plenty of simple ways to clone/rotate rsync-created backups (even single-file ones) ... that problem was solved a long time ago, thankfully. –  user227963 Jul 1 at 17:17

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.