Setting up a dense virtual lab [closed]

Question

I want to create a lab of virtual machines for testing. Something on the order of 80 Windows machines and 20 Linux boxes. The client OS doesn't need to do much but respond to ping, etc. They can run slow. I'm wondering if it's even reasonable to hope to get something like that on a single beefy physical box. I've been reading about XP that can run in 80MB, so 80x100 = 8GB -- at least that seems reasonable. Disk space wouldn't be an issue. A dual-quad Intel i7 might be up to the job???

I'm guessing VMWare ESX is the way to go, but I'm open to suggestions. If I can manage to get 1 simple XP image, and a Linux image, can ESX duplicate it that many times?

Can a single ESX server be loaded up that dense? Any suggestions?

Answer 1

I think VMware is your solution here - you'll want ESXi. Although VMware "technically" supports up to 128 machines per physical server, I've seen a demonstration where they had 178 running! So this won't be an issue.

Depending on the performance requirements, here's what you'll want to do: Get yourself a beefy server. At least dual-quad i7's. Also, go for a configuration of about 96GB RAM. You can halve that if you only need barebones performance, but really don't shoot yourself in the foot and go with too little RAM - it's not that expensive these days. With the i7's running optimized with RAM in triplets, this is an ideal configuration for performance. You can assign each VM 1GB, which should be adequate for your testing environment. Also, you'll want maybe 4 drives in a RAID 10 for performance. You can thin-provision which is nice - assign each machine maybe 20 GB for the install, but it will only take the space needed for the actual files - maybe 2-3GB per server max. Times 100 is about 300GB - go for 4x250GB drives and you'll have the space, performance and redundancy you need.

This server shouldn't cost more than about $7000, depending on your RAM choices. The full 96GB might be a bit more. I'd recommend Dell on this one - they seem to have the best price/performance ratio right now. You won't need extended warranties or business critical support - NBD would be adequate.

The VMware ESXi will be free for you in this case - you don't need anything that would require a license here. Just register for the free license.

The OS license gets a little trickier. I'll leave it to you, but probably you'll want Server 2008 Datacenter, licensed for 2 processors. It's about $4000/processor. This allows an unlimited number of Windows virtual servers to run on the box, and they can be any flavor of windows (Server 2003, 2008, Standard, Enterprise - it doesn't matter) For the Linux boxes it's no issue.

For the machines - the Windows boxes are easy enough. VMware has a cloning tool that works like a disk clone utility. Set up your first server, then prepare it - use Sysprep - then shut it down and clone it. You know, 80 times. Once that's done, fire up each server one at a time, run through the prep tool to rename and whatnot, and you should be good. For the Linux boxes I'm less helpful - you can clone them of course, but after that I'm not much use - I'm a Windows guy.

Total cost here with Licensing is about $15,000. Would that work?

Answer 2

VMware ESX would be perfect for this. You'll want to get the Virtual Center as well so that you can create a template then deploy all the VMs from the Template. You can do it manually without Virtual Center but it's a lot harder.

Yes a single ESX server can handle that many VMs. Just get a server with 16 or 32 Gigs of RAM, and you'll be fine.

Answer 3

Even if XP will run with 80MB of guest ram (which is doubious), no VM layer I know with run 80 VMs on 8 cores will. The I/O crunch generated 80 VMs booting is too awful to think about. Consider carefully that XP will probably want to use swap space if you dial down the guest RAM. Guest I/O is fatal in virtualized environments. Perhaps you get get the XP boxes to PXE boot from fast disk system? That might help.

I recommend splitting that 80 across at least boxes of the spec you suggest. You'll be a lot happier, but don't expect snappy performance from any of the VMs (but do consider running XP with 128MB). Even in this setup, you'll have to watch your I/O load. Put the VMs on an iSCSI system too.

ESX is a good way to go here, but you might get that kind of performance from Citrix Xenserver too. I think VMware Server and VirtualBox are out of the running unless you use more boxes.

Hope this helps.

Answer 4

Are you sure that "The client OS doesn't need to do much but respond to ping, etc." if so then look at setting up multiple ip addresses on a single machine ( or two if you need to route packets between them ).

Answer 5

A single ESX or ESXi system will easily hand this load. I have a pair of DL580 G5's with 64GB of RAM each, handling a total of about 100 fairly active development VMs.

A HP DL380 G6 or equivalent with 64GB of RAM is priced under 10K list these days and is roughly the power of a DL580 G5.

If you have a storage array that supports deduplication (NetApp, EMC etc) then you may not need very much storage, maybe 50GB total might do your needs. Of course you can put together a cheap NFS array with Red Hat if you wish but it can't handle the dedupe AFAIK.

One thing to remember is that if all/most the VMs are running identical code then ESX will be able to really take advantage of it's memory sharing technology. The hosts I mentioned above are able to shed 50% of their RAM utilization due to ESX's memory sharing and they are running a fairly mixed load of 2000/XP/2003/2008/Red Hat. AFAIK only ESX has the memory sharing technology.

Answer 6

I would check out ESXi if you only plan on setting up one host. It's free and easier to set up than ESX.