13

If I am running a modern vmware ESXi system, I can drop in a statically linked rsync binary and rsync files to any destination over SSH.

I'm trying to understand why most (all ?) backup of vmware guests is not done this way.

If the VM is running, you can simply use 'vim-cmd vmsvc/snapshot.create' to create a snapshot and then rsync that snapshot to the remote host. (there's even an option to "quiesce" the snapshot)

OR, if you want a more robust backup, you can gracefully halt the VM and rsync over the vmdk file(s).

So ... it seems like I am a simple shell script away from all the backups I ever wanted to do, simply and easily, using plain old rsync.

What am I missing here ?

Improve this question
6
  • 1
    Because if a single file changes in the VM you will have to backup the whole vmdk?
    – faker
    Commented Jun 25, 2014 at 22:14
  • No, rsync will update a single file efficiently with just the changes since the last transfer. Certainly the operations of the VM could produce a LOT more changes than you expect it to, but it's not going to make you resend the entire vmdk ...
    – user227963
    Commented Jun 25, 2014 at 22:47
  • Other than the fact you shouldn't use the esxi shell for anything other than maintenance, the esxi OS is not made to work in that way, and you would be unsupported, I think you are misunderstanding the concept of a snapshot. The snapshot in this case is a delta. So if you take a snap and copy it straight away, it would be tiny, and contain almost no information. You are thinking of a backend storage snapshot, and yes people back up VMs in this way
    – Rqomey
    Commented Jun 27, 2014 at 12:08
  • 1
    @Rqomey - there are different kind of "snapshots" in ESXi. You're talking about the one kind that is visible via the vSphere Client - but using the API you got other options, e.g.: full clone.
    – omni
    Commented Jun 27, 2014 at 15:59
  • @MASI Do you mean a clone then as opposed to a snapshot? ;)
    – Rqomey
    Commented Jun 30, 2014 at 10:27

4 Answers 4

Reset to default
35
  • Because the transfer speeds out of the ESXi console are purposefully limited.
  • Because this isn't scalable in any way.
  • Because you'd have to drop a statically-compiled rsync binary onto the ESXi host.
  • Because the VMs, the VMDKs, their ramdisk files and other components can change enough to make rsync a losing proposition... do you really want to re-sync a 200GB VM that was rebooted and had a small number of files change?
  • Because of CPU/memory resource requirements on the source or destination. Rsync isn't free.
  • Because there are other products on the market, both third-party and VMware-provided. Look up Changed Block Tracking.
  • Because ESXi is NOT a general-purpose operating system.

Also see: Install rsync on VMware ESX 4.1 server

Improve this answer
12
  • 1
    Outstanding answer.
    – EEAA
    Commented Jun 25, 2014 at 22:24
  • 4
    They aren't... I mean, it's in the name: ghettoVCB. There are better solutions out there. Veeam, vSphere Data Protection, etc.
    – ewwhite
    Commented Jun 25, 2014 at 22:34
  • 2
    You could certainly use the rsync method if you switch over to xen/kvm.
    – Zoredache
    Commented Jun 25, 2014 at 23:01
  • 9
    @user227963 Rsync is also rather inefficient on both - large number of files as well as large files. And while it might not have to resend the entire file over the wire, it will have to reread it on source and destination alike. CBT will help you here, but rsync knows nothing about CBT.
    – the-wabbit
    Commented Jun 25, 2014 at 23:04
  • 2
    @user227963 copying files is simple. Now make it fast and not a resource hog on large files with small constant changes. rsync is decent but nowhere near the performance of anything with insider info on which blocks changed.
    – JamesRyan
    Commented Jun 26, 2014 at 9:29
4

I used to do just this a few years back. (edit: with VMWare running on CentOS hosts, not ESXi admittedly)

Every night I had a script that would suspend a VM, rsync the files from disk to the backup server and then start the VMs again. It worked quite well except...

Rsync doesn't work very well with a 2GB file.

Its not because rsync isn't brilliant, it more that each 2GB vmdk file changes in ways that are very opaque to rsync, even small changes to the enclosed filesystem produce changes in the vmdk (or all vmdks for some reason) which I blamed on Windows, either automatically defragging or otherwise doing all the other things it does that don't matter if you're running a real system, but show up when you are trying to rsync a VM!

I think the rsync mechanism for detecting changes don't work very well on a 2GB file, whilst it quite often skipped chunks of the start of the vmdk, once it started to find a difference it would simply copy the rest of the file. I don't know if that's an issue with rsync not being able to detect a moved chunk of binary data, or with a lack of memory on the source box, or whether the vmdk just got updated all the way through. It doesn't matter as the result wasd the same - majority of the vmdk got copied.

In the end I simply copied any changed files and overwrite them, still using rsync. I also had better performance simply overwriting the backup file instead of letting rsync copy and replace what was there.

Our backup server wasn't the fastest either and it got to the point where overnight wasn't long enough to back up all running VMs.

However, when we did need to restore a VM, it was really easy and worked beautifully.

Improve this answer
3
  • 1
    Ok, that's very helpful. I know a bit about how rsync works, and I can tell you it has nothing to do with the size of the file - but what you are describing is that much more of the file changes than you expect it to ... that is to say, you run the VM for a day, and you only do a few little things with it, and then you stop it ... but the vmdk file changed by 30-40% (even though you did very little). So rsync would do just fine, it just has a lot of work to do ... more than you expected. Thanks!
    – user227963
    Commented Jun 26, 2014 at 16:11
  • 1
    But then ... the question this raises ... how do the "professional" tools do it ? What kind of magic are they doing that is somehow more optimal than what rsync (or scp, or even cp) would do ? At the end of the day, you have a unix environment (the ESXi console) and you want to move a file in or out of it ... what secrets could there be involved with that ?
    – user227963
    Commented Jun 26, 2014 at 16:14
  • @user227963 The professional tools leverage features like the changed-block-tracking or have access to other vSphere or ESXi APIs.
    – ewwhite
    Commented Jun 27, 2014 at 12:44
3

Rsyncing a single file is not a backup solution,

what do you do when something happened to the vm and files were deleted, but you only noticed this after your rsync has run again? You will have overwritten the good 'backup' of your files with the bad image now.

If you want backup you need to keep the old versions somewhere, or the diff's. Rsync will only copy over the diffs for you, but it will not store only the diff's, but overwrite the previous file.

There might be options for you here, with rsync, and a copy-on-write filesystem with versioning information, which will in effect store the diffs every time your rsync script runs. This solutions starts to get a bit more complicated already, so this is why people resort to known working solutions imho.

Improve this answer
1
  • There is certainly a lot more complexity involved here than I originally thought, but what you're mentioning is not a problem. Certainly if you blindly ran rsync over and over you would run into trouble, as you suggest, but there are plenty of simple ways to clone/rotate rsync-created backups (even single-file ones) ... that problem was solved a long time ago, thankfully.
    – user227963
    Commented Jul 1, 2014 at 17:17
2

There isn't any reason why you can't use Rsync in an ESXi server. We offer a statically compiled version here https://33hops.com/download-form.html that works very well (just unfold the software list). There's information on how to compile your own too in the link below.

Nevertheless anybody willing to use it must take on account that Rsync and its Delta algorithm weren't thought to backup huge fixed lenght sparse files, like VM hard disks, but to sync variable length smaller files. So, it does work, but it takes a lot of time and CPU to calculate the diff data. In fact it's just a way to exchange bandwidth by CPU. In any case, it's still quite workable, specially if your virtual disks are in the order of a few tens of gigabytes.

I have published a full post on the topic here, detailing all pros and cons Rsync Considerations copying ©ESXi VM

Improve this answer
2
  • 1
    That's very kind, but that version crashes with out of memory in receive_sums [sender] on large files… 3.1.1 seems to fix it.
    – Pierre-Luc Paour
    Commented Apr 13, 2020 at 7:44
  • Well, as stated, Rsync wasn't though to sync huge files, we offer other binaries specially designed for that purpose. You have all the details to compile a static binary from some more recent source code, in any case using Rsync to sync virtual disks over some tens of gigabytes in size can be really slugish
    – Daniel J.
    Commented Dec 6, 2020 at 17:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .