24

Standalone ESXi (4.1) host without any vCenter Server.

How to backup virtual machines as quickly and storage-friendly as possible?

I know I can access the ESXi console and use the standard Unix cp command, but this has the downfall of copying the whole VMDK files, not only their actually used space; so, for a 30-GB VMDK of which only 1 GB is used, the backup would take 30 full GBs of space, and time accordingly.

And yes, I know about thin-provisioned virtual disks, but they tend to behave very badly when physically copied, and/or to blow up to their full provisioned size; also, they are not recommended for actual VM performance.

It is ok for me to shut down the VMs before backing them up (i.e. I don't need "live" backups); but I need a way to copy them around efficiently; and yes, a way to automate shutdown/startup when taking a backup would also help.

I only have ESXi; no Service Console, no vCenter Server... what's the best way to handle this task? Also, what about restores?

8 Answers 8

21

My preferred solution for this is to simply export them to an ovf or ova file using either the vSphere client or the command line ovftool.

In the vSphere Client, make sure the VM is off, then highlight it and go to File->Export->Export OVF Template. Then just follow the prompts.

Restoring is a piece of cake, just do the reverse (the menu option is "Deploy OVF template", I think).

To create a thin backup using ovftool

ovftool -dm=thin  vi://<user>@<esxi-host>/<vm-name> <local-file>.ovf

You may also wish to check out some of the options at http://www.virtuallyghetto.com/, I know these are very popular and I think there are some good choices for backups, although I haven't looked at any of them too recently.

2
  • OVF backup and restore seems to work great as a poor man's VMWare backup option. Eventually I intend to automate using the cli ovftool, but shutting down the vm inside the script while the backup runs will be the tricky part. Nov 24, 2015 at 16:33
  • OVF is very slow. If you want to avoid downtime, you can use the web client to clone a running VM with paying for vMotion, then shutdown and backup the clone. No downtime. May 12, 2017 at 2:22
7

I ended up writing a script which copies the VM configuration files and uses vmkfstools -d to clone the VMDKs while preserving the thin provisioning.

For reference:

#!/bin/sh

if [ $# != 2 ]; then
        echo "Usage: $(basename $0) <SOURCE VM PATH> <DESTINATION PATH>"
        echo "Example: $(basename $0) /vmfs/volumes/datastore1/VM1 /vmfs/volumes/datastore2"
        exit
fi

vmx=$(basename $(/bin/ls $1/*.vmx))
name=$(grep displayName $1/$vmx | /bin/awk -F\" '{print $(NF-1)}')
vmxf=$(grep vmxf $1/$vmx | /bin/awk -F\" '{print $(NF-1)}')
nvram=$(grep nvram $1/$vmx | /bin/awk -F\" '{print $(NF-1)}')
vmdks=$(grep vmdk $1/$vmx | /bin/awk -F\" '{print $(NF-1)}')

echo "Started copying VM $name"

vmdir=$(basename $1)
destpath="$2/$vmdir"

echo "Source path: $1"
echo "Destination path: $destpath"

echo "Creating destination path $destpath"
/bin/mkdir -p $destpath

echo "Copying configuration files:"
echo $vmx
/bin/cp $1/$vmx $destpath
echo $vmxf
/bin/cp $1/$vmxf $destpath
echo $nvram
/bin/cp $1/$nvram $destpath

echo "Copying virtual disks:"
for vmdk in $vmdks;
do
        echo $vmdk
        /sbin/vmkfstools -d thin -i $1/$vmdk $destpath/$vmdk
done

echo "Completed copying VM $name"

This requires the VM to be powered off and have no active snapshots.

2
  • Hi, what about thick lazy?
    – rovshango
    Sep 9, 2015 at 19:22
  • Great script @Massimo, but I had to add double quotes to vmx assignment to avoid basename syntax error: vmx=$(basename "$(/bin/ls $1/*.vmx)") . But now, if $1 path contains spaces, it crashes because $1 is not quoted. I could not get rid of this problem. Any idea?
    – Blazeag
    Dec 14, 2017 at 18:28
5

I don't know if this fits the bill for you, but VM Explorer does a nice job of performing hot or cold backups of virtual machines. I believe that with ESXi 4.1 VM Explorer allows you to perform VM guest backups from one host to another host as well.

3
  • I will have to look at this more but right from the start I see something I like - Licensed per installation.
    – Chadddada
    Sep 16, 2011 at 13:33
  • The basic version is free Jul 5, 2016 at 7:48
  • 3
    Good to note that this is no longer free, the lowest tier is "Professional" at $699 Jan 8, 2018 at 3:21
4

I would try to use a proven solution of some sort rather than roll your own. The reduced hassle, time, and risk will readily pay for itself even if you opt for a solution that isn't free. All of these issues you are concerned about are addressed in any modern backup solution for ESXi.

The solution that a client uses in their ESXi environment with good luck is Veeam. There is even a free edition that may work for your needs: https://www.veeam.com/virtual-machine-backup-solution-free.html

2
  • 1
    Free edition still needs ESXi licensing.
    – Overmind
    Mar 1, 2018 at 11:31
  • @Overmind You're absolutely correct. I overlooked that it doesn't work (ironically, perhaps) with the free vSphere Hypervisor edition. Apr 5, 2018 at 6:00
3

XSIBackup is a highly optimized tool programmed in C and assembly that is offered in two editions to cover all possible needs:

  • XSIBackup-DC (download here): a service running in the ESXi hypervisor to backup standalone hosts. It needs no additional elements, it can backup/replicate the VMs to local storage or over IP to any IP/SSH Linux host.

  • XSIBackup-App (download here): an appliance based in CentOS 7 that can be easily deployed to any ESXi box. It allows to connect to any number of ESXi hosts to backup/replicate their VMs from a centralized console to any IP/SSH server or to virtually anything that can be mounted in Linux. That includes most Cloud service providers.

Both of the above editions offer a unified console plus an nCurses GUI that can be used over a regular SSH connection. It offers almost any feature that you could expect:

  • Compatible with ESXi 5.1 to 8.0
  • Hot backup/replication of Virtual Machines.
  • Block level deduplication + LZJB compression, up to 99% compression ratio.
  • Delta algorithm, zero awareness and CBT native support even on ESXi free.
  • Full or granular restores by using the command line tool xsigr.
  • E-mail report layout, fully customizable.
  • Local or over IP/SSH backups.
  • Multiserver backup (XSIBackup-App).
  • Fully scriptable to integrate in wider schemes.
  • Highly stable. It's a C binary running in Linux.
2

Ghetto VCB can do the backup while the machine is running. For the space you can use a deduplication+compression filesystem like lessfs on the backup server.

0
0

I'm exporting to OVF format, too. It's fine because it compress on the fly the image with gzip.

But the licenses are gone, because with importing an OVF file into an ESXI a machine with new hardware is created.

Solution:

I backup (download) the file folder of the virtual machine without the big image and compress them.

Both - ovf folder and compressed copy of virtual image files (without big image) - are in the same folder.

Restoring:

-> Importing OVF into ESXI and upload the uncompressed config (from backuped file folder) -> Virtual Machine is fine again

ismail

0

cpio can copy sparse files while preserving the "holes" in them.

1
  • AFAIK, there wasn't any tool that could handle sparse.vmdk files natively, apart from vmkfstools, which is limited to copy files within a local file system. That's why we created (c) XSIDiff as part of our suite of tools for ESXi. It copies only used blocks as vmkfstools does, but you can copy data over SSH and also as a TCP/IP client/server app. You can download a free working version here: 33hops.com/download-xsidiff-trial.html
    – Daniel J.
    Aug 9, 2017 at 17:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .