Virtual Machine Count planning
When it comes to figuring out how many VMs you should plan for on a single host, there are actually no really good rules of thumb. In fact, there is only one, and it is only kinda good:
Virtual-Machine counts are usually bounded by RAM, except for when they're not.
Which isn't terribly helpful. If those VMs are going to be running low-CPU applications, then your limiter is going to be based on RAM. Each VM platform has it's own abilities to oversubscribe RAM, so it isn't as easy as TOTAL_RAM / Per-VM-Ram = MachineCount, but that number is a good planning item.
But what if your VMs are doing things besides low-CPU packet-slinging?
Virtual-machine counts are bounded by 7 discrete resources available to the host machine:
- Hypervisor VMware, Xen, HyperV, KVM, whatever. Each has their own count-impacting features. Some are very good at memory-page deduplication, others not so much. Some don't permit oversubscription of CPU capacity, some do.
- CPU Core Speed This limits the maximum single-threaded performance a VM will be able to run. 36 cores of 1.8GHz CPU may be 64.8GHz of CPU on a host, but no single thread will run faster than 1.8GHz.
- CPU Core Count This, with core-speed, describes the ceiling of maximal CPU performance you can experience.
- System RAM As described above, this limits the number of VMs you can run. Certain hypervisors are better than others at things like memory-page deduplication, so if you're running 100 identical VMs you can pack a lot more of these on such deduplicating systems than if you were running 100 completely different VMs.
- Disk Size Each OS image takes a certain amount of space. You need enough space to store it all. Therefore, disk-size puts an upper limit on how many VMs you can host.
- I/O Bandwidth The disk underlying the VMs has a maximum on how many I/Os per second it can handle. If you throw too much at it, systems will bog down waiting for the I/O to complete. This puts an upper limit on how many I/O consuming VMs you can run.
- Network Bandwidth For network-using VMs, the available network bandwidth will put a ceiling on how many such VMs you can run on a given host.
All of these can be the thing you trip over, it all depends on what you're doing with your VMs. Some things to remember:
- There is no such thing as a generic system.
- There is no such thing as a generic web-server, since application code can run from barely-moves-the-needle CDN-style serving, to big deep-crack stuff like video transcoding.
- There is no such thing as a generic database server. These can run from tiny systems used just for session-state-tracking, to very big ones.
To figure out how many VMs you can pack into a host-system, you need to know how your systems run and what they require to run well. Once you know that, you can then do the count-planning. And better yet, figure out how beefy you need to make your host-systems!