9

Does it make any sense to run a virtual machine with a "master" guest,
and in that master guest run many other guests?

Has anyone tested this? Is it even possible?
Are there better ways to accomplish my goals? (Read on.)
(I googled for "guest in guest" and found nothing)

What I want to do: I'd like to set up and test various virtual networks, with database and application servers, and test different firewall, DNS server, database server configs, etc.

I could do this by running guests directly on my host computer, but then I might need to modify the network config on my host (e.g. configure dnsmasq?), depending on what network setup I'm about to test. I'd also need to start each guest individually. Whereas if all guests run inside a master guest, then I can config the network on that master guest, and I need only start and stop that master guest, and take snapshots of it, to implicitly start & stop & snapshot all guests running inside it.

(I would have many master guests, probably only one running at a time.)

I'm using Ubuntu 11.4 and KVM, with hardware virtualization support (AMD-V).
Do you think the guests-in-the-guest would benefit from virtualization?

Update: I should have googled for "Nested Virtualization" not "guests in guest". Now I find lots of links :-) E.g. this example of actually migrating a running VM from the host machine to a nested VM host: VMotion from physical ESX 4 to virtual ESX 4
You who replied, thanks for mentioning "Nested Virtualization" :-)

Update: Performance info, for AMD processors, from 2008: [...] up until now, when kvm virtualizes a processor, the guest sees a cpu that is similar to the host processor, but does not have virtualization extensions. This means that you cannot run a hypervisor that needs these virtualization extensions within a guest (you can still run hypervisors that do not rely on these extensions, such as VMware, but with lower performance). With the new patches, the virtualized cpu does include the virtualization extensions; this means the guest can run a hypervisor, including kvm, and have its own guests. (thanks "wzzrd")

Update: Performance info on Intel processors, from 2011: "There ist no support for nested virtualization with intel vmx in the current version of kvm in the Ubuntu repositories. With the newest patches for kvm it is possible, but there are still in development"

2
  • 3
    Keep a spinning top handy.
    – nedm
    May 27, 2011 at 17:19
  • Nested virtualization is possible, but you double your CPU-related performance penalty, waste RAM because in the default configurations, both the host, master guest and "guest" guests will each other cache the same data multiple times, and IO performance will be disastrous for data that wasn't cached. Oct 6, 2015 at 0:42

8 Answers 8

3
  1. Its possible, read more here - http://blog.jasonruiz.com/2011/01/24/kvm-nested-virtualization-support/

  2. the current purpose of nested virt is testing, but I think it meant to test hypervisors not databases, dns, etc...

in order to test such services I think "normal" virtualization will do.

you can have a few different virtual networks, with different dhcp/dns/routing services.

0
7

What you want to do is possible, it's just not really practical from a performance point of view.

1
  • 2
    Concise, to the point and true :)
    – wzzrd
    May 27, 2011 at 11:16
6

ESXi can virtualize itself. So you can run ESXi as a guest on ESXi (or ESX for that matter).enter image description here

Directions found here

0
3

I don't know about other hypervisors, but it's definitely possible with ESX/i, as discussed here. Also, it's definitely not possible with Hyper-V, as discussed here.

Howewer, where this is possibile, it's totally unsupported and also quite useless, unless for testing purposes. I personally did it with ESX to test a VMware cluster, which required at least two ESX hosts, a Virtual Center machine and some shared storage; I didn't have more than one server nor any "real" storage available at that point, so I used virtual ESX hosts with a shared virtual disk (just like I would have done to test a Microsoft cluster). It worked, and it was also not terribly bad at performance. But I can't think of any reason in the world to use this kind of setup in a production environment.

1
  • 1
    This was an interesting example. I'd use nested VMs for testing too, like you, not for production.
    – KajMagnus
    May 29, 2011 at 14:08
0

kvm within kvm will not work, though the inner guests can run on pure qemu, in emulation mode.

I'd leave the VMs on a single host, without overcomplicating things

4
  • Untrue. It is very possible and has been since 2008 or so. Link: linux-kvm.com/content/kvm-nested-virtualization-works
    – wzzrd
    May 27, 2011 at 11:15
  • 1
    yes, with very specific CPU instruction sets, and nowhere near production or serious testing environments. The T/S wants to test network configs and services, not additional hypervisors, so why complicate things this far? Especially since the standard management tools. You're right on the account of kvm within kvm can work, under certain conditions, that was my mistake, but to answer the original question - this is not the best idea under any hypervisor
    – dyasny
    May 27, 2011 at 11:31
  • "this is not the best idea under any hypervisor": now that is true :) here, have an upvote :)
    – wzzrd
    May 28, 2011 at 20:01
  • LOL, not that an upvote is that important :)
    – dyasny
    May 28, 2011 at 20:27
0

KVM probably is not the best option for your purpose, you should try Virtualbox.

With Virtualbox, you can make a virtual machine inside a virtual machine, and the network tools are very convenient.

5
  • 1
    Rubbish. KVM will suffice perfectly.
    – wzzrd
    May 27, 2011 at 11:16
  • @wzzrd I didn't say KVM is not suffice. I said Virtualbox may be a better option, in this case, since it's a complex configuration.
    – erickzetta
    May 27, 2011 at 11:48
  • Actually, I like configuring KVM VMs with virsh :-)
    – KajMagnus
    May 27, 2011 at 16:17
  • support for VMX, second level address translation (SLAT), para-ops, nested pages, machines within machines--this was a feature introduced way back in the 2.0 major release of VirtualBox. virtualbox.org/wiki/Changelog-2.0 Aug 11, 2014 at 0:41
0

Have you checked out XenServer? The new beta version is called "XenServer boston" Read more about it here:

http://blogs.citrix.com/2011/05/18/xenserver-boston-beta-program/

In the release notes, you will see the following:

Virtual Appliance support. Within XenCenter you can create multi-VM virtual appliances (vApps), with relationships between the VMs for use with the boot sequence during Site Recovery. vApps can be easily imported and exported using the Open Virtualization Format (OVF) standard

Using this feature combined with Virtual Lans, you will be able to accomplish what you want. I'm not positive, but I believe you can also create a snapshot of all the Vm's in the Virtual Appliance at once.

Please let me know if you have any further questions about XenServer (I use it 2x every day)

-1

Whether it's possible or not is irrelevant. It just doesn't make sense. You gain absolutely nothing by nesting VMs, while losing a lot.

5
  • Actually, the/some KVM developers think it makes sense: on the KVM dev mailing list, there's this thread with a patch for nested virtualization support. I think the developers seems fairly happy, if you continue reading some replies.
    – KajMagnus
    May 27, 2011 at 16:01
  • A non-KVM developer who likes nested virtualization: "Another use is to have multiple virtualized networks inside one machine. Each network would have a different configuration of hosts. I've been waiting for this." (From here.)
    – KajMagnus
    May 27, 2011 at 16:02
  • @KajMagnus, just because people want it doesn't mean it makes sense. Some people simple can't see the wood for the forest. May 27, 2011 at 23:25
  • Makes sense for testing scenarios. Want to deploy a test hypervisor deployment just to see how a particular feature behaves in a new version? Spin up a VM. You're correct that it's not much use for OP's particular usage, though. Aug 15, 2011 at 23:31
  • The IBM VM folks used to do this all the time when developing new versions of VM. There's a great paper on it (eg p. 28), if you're interested.
    – fission
    Dec 5, 2012 at 7:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .