So, I've got a pretty awesome KVM setup running now, CentOS5.5 on the host and guest domains, libvirt managing all the configuration, etc. The guest domain filesystems are stored in LVM on top of a hardware RAID5 volume, so I have flexibility for backups and low-level data redundancy.
I tested out virt-clone today, and it worked amazingly well, except that it took about 30 minutes to replicate 24G of data from the suspended domain's LVM disks to the new LVM volumes for the new virtual machine.
My question is: couldn't I just use an LVM snapshot to create the new VM's root disk? For example:
lvcreate -s guest1_root -n guest2_root -L 8G raid_vg
Now, my understanding of LVM snapshots is that the snapshot stores an inverse delta of the changes made to the original blocks so that the snapshot takes up little actual space and the original blocks can be read out even after the original volume has been written to. LVM2 adds read-write snapshots, which opens up this interesting possibility.
Indeed, the LVM HOWTO even suggests using this feature in conjunction with Xen:
This opens up many new possibilities that were not possible with LVM1's read-only snapshots. (...) It is also useful for creating volumes for use with Xen. You can create a disk image, then snapshot it and modify the snapshot for a particular domU instance. You can then create another snapshot of the original volume, and modify that one for a different domU instance. Since the only storage used by a snapshot is blocks that were changed on the origin or the snapshot, the majority of the volume is shared by the domU's.
This seems like a really powerful tool, and I'm wondering if anyone has tried it in their production virtualization environment, can think of any vendors (Citrix, VMWare) that take this approach, or can think of any serious problems with this idea. I can think of the following potential problems:
- Potential filesystem problems if the "original" guest is running at snapshot time.
- Host domain LVM performance as the "inverse delta" and "written block" logs are appended to. It may be very fast, but I don't know, and this seems like a weird idea, so it bears testing and benchmarking.
- More eggs in one basket: if the "original" LVM volume gets corrupted, then the snapshotted volumes are hosed as well. Of course, RAID5 mitigates disk corruption, but it's worth noting.
- Can you snapshot a snapshot? I assume so.
As people who likely have more virtualization experience than I, is there anything that makes want to run screaming from such a setup?