Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

What's the easiest/quickest way to clone a VM from one server onto another if you don't have shared storage between the two servers (so you can't do a standard migration)?

I have a production ready VM installed on one server, and I want to clone it onto another system. I don't have shared storage between the two hosts, but I've copied the disk image between the two hosts and added a config for it (virsh defined it). When I try to start it however it doesn't take:

# virsh create /etc/libvirt/qemu/cloned-vm.xml 
error: Failed to create domain from /etc/libvirt/qemu/cloned-vm.xml
error: Unable to read from monitor: Connection reset by peer

I'm using KVM on RHEL6. Here is the duplicated config

<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE 
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
  virsh edit 
or other application using the libvirt API.
-->

<domain type='kvm'>
  <name>cloned-vm</name>
  <uuid>NEW_UUID_HERE</uuid>
  <memory>15360000</memory>
  <currentMemory>15360000</currentMemory>
  <vcpu>7</vcpu>
  <os>
    <type arch='x86_64' machine='rhel6.2.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/local/vm/cloned-vm.img'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <interface type='bridge'>
      <mac address='NE:W_:MA:C_:AD:DR'/>
      <source bridge='br2'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <input type='tablet' bus='usb'/>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
</domain>
share|improve this question
    
Do you want to do offline migration, or just shutdown and boot? –  mgorven Jun 18 '12 at 16:38
    
The VM to duplicate isn't in production, so it doesn't matter to me, I just need it duplicated across servers. –  Andrew Case Jun 18 '12 at 16:49

2 Answers 2

And here is what I do where $VM is the VM name, and DEST is the destination hypervisor hostname. It does this on running VMs using dd and snapshot LVM disks (It is assuming that the LVM Group is called HypGroup00).

I just threw this together, so it isn't necessarily the prettiest, but does the job, I'm using it to migrate some VMs from a CentOS 5.9 hypervisor to CentOS 6 with minimal downtime.

This is for CentOS 5.9, tested on going to CentOS 5 and 6 as a destination.

VM=webserver
DEST=hyp5

Figure out which disks to copy, since most of our VM disks are referenced via the /dev/mapper/ path rather than the /dev/volgroup/volname path. So we need to translate between the two.

DISKS=`cat /etc/libvirt/qemu/$VM.xml |  grep HypGroup00 | sed "s|.*/\(HypGroup00-.*\)'/>|\1|"`

LVS=""

for disk in $DISKS; do
  echo VM Disk $disk
  LV=`lvdisplay /dev/mapper/$disk | grep "LV Name" | awk '{print $3}'`
  LVS="$LVS $LV"
done

Copy over the VM definition and register it

virsh dumpxml $VM > /tmp/$VM.xml
scp /tmp/$VM.xml $DEST:/tmp/
ssh $DEST virsh undefine $VM > /dev/null 2>&1
ssh $DEST virsh define /tmp/$VM.xml

Now create the LV on the remote server

for lv in $LVS; do
  ssh $DEST lvremove --force $lv
  SIZE=`lvdisplay $lv | grep "LV Size" | awk '{print $3}'`
  SHORTNAME=`basename $lv`
  ssh $DEST lvcreate -L"$SIZE"G -n$SHORTNAME HypGroup00
done

Now create the snapshot LVs and start copying over the data

virsh suspend $VM
for lv in $LVS; do
  lvcreate -L10G -s -n $lv-snapshot $lv
done
virsh resume $VM

Copy the disks across

for lv in $LVS; do
  echo Copying LV $lv, this will take a while...
  time dd bs=1M if=$lv-snapshot | gzip --fast | ssh $DEST "gzip -d | dd of=$lv"
done

A more complex version of the above that shows progress if required from dd, no good for multiple copies because of /tmp/pid, but you can change to to include $$ if you wish.

(dd bs=1M if=$lv-snapshot & echo $! >&3 ) 3>/tmp/pid  2> >(grep 'copied' 1>&2) | ssh $DEST "dd bs=1M of=$lv" &
# Need this sleep to give the above time to run
sleep 1
PID=$(</tmp/pid)

while kill -0 $PID; do
  kill -USR1 $PID
  sleep 5
done

Clean up

for lv in $LVS; do
  lvremove --force $lv-snapshot
done
share|improve this answer
up vote 2 down vote accepted

Okay so the way I was doing it actually did work just fine. The problem is just that I didn't have enough resources to run that VM. So just to answer my own question... here are the details to how I did the VM duplication across different servers without shared disk.

Because you don't have a shared disk, you can't do a typical 'clone' and then 'migration'. Instead you do a typical clone

  1. Here's the command to do the cloning (/local/vm/ is the path to your VM images, usually /var/something/):

    virt-clone --original=vm-to-clone --name=cloned-vm -f /local/vm/cloned-vm.img --mac=xx:xx:xx:xx:xx:xx

  2. Now copy that img file from one server to the other... my servers can't talk directly to eachother, so I use this little SSH redirect to do the trick:

    ssh -n server1 '(cd /local/vm/; cat cloned-vm.img)' | ssh server2 '(cd /local/vm/; cat > cloned-vm.img)'

  3. Then copy the config for that VM over:

    ssh -n server1 '(cd /etc/libvirt/qemu/; cat cloned-vm.xml)' | ssh server2 '(cd /etc/libvirt/qemu/; cat > cloned-vm.xml)'

  4. Update the config with any new changes. In my case (and this was what was causing my problem), I needed to lower the "memory" and "currentMemory" attributes.

  5. Add the new VM to libvirt:

    virsh define /etc/libvirt/qemu/cloned-vm.xml

  6. Run it:

    virsh create /etc/libvirt/qemu/cloned-vm.xml

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.