up vote 4 down vote favorite
share [g+] share [fb]

What are the minimum requirements to boot a KVM instance. I want to boot 1000 instances all at the same time and I am using a NAS environment. I want to make sure that I know what the IO requirements are for a base boot.

link|improve this question
6  
1000 instances at once? You had better be prepared to fork out some huge coin to support that many clients. – ErikA Dec 15 '10 at 16:54
Do you really, no really, need 1000 instances at once? – AliGibbs Dec 15 '10 at 17:08
1  
Are these identical instances? If so, and if you cache very agressively, you stand a good chance of the boot image being read out of RAM. That would take care of a lot of the disk IO bottleneck, but then you need to ensure that your network/SAN is able to keep up with the load, which in this case, would need to be comprised of some very high-performing 10GB switches and HBAs. – ErikA Dec 16 '10 at 1:59
I suspect the usage of ksm might help, when instances share the same memory pages. If the instances are identical this could reduce I/O for code. – AndreasM Aug 3 '11 at 13:57
feedback

5 Answers

up vote 5 down vote

Boot storms are something that shouldn't be taken lightly as they can cause seriously big storage arrays to grind to a halt. Avoiding these sudden boot storms due to that many instances starting at once should be a major design consideration.

How many IOPS are really needed is difficult to model and you have to measure on your own. You have to chose the toolbox according to the NAS you have for that. Be more specific if you can .

There's also a heavy dependence on what you are actually booting up there.

link|improve this answer
feedback
up vote 2 down vote

It depends on the VM obviously - just booting GRUB? maybe 50-100 operations - an exchange server? many, many more...

The answer is easy, benchmark one - it's the only way to know.

Oh and does it have to be KVM? Something like that sounds more like the kind of thing VMWare's View product was built for.

link|improve this answer
feedback
up vote 1 down vote

Some comparative test data (200 VMWare VMs) here: http://ctistrategy.com/2009/12/28/vmware-boot-storm-netapp-part-2/

Cheers

link|improve this answer
1  
Just for reference the fairly small 16GB PAM modules referenced in this cost $15k each, and newer (but bigger ones) cost more. You'll hopefully be installing two as I'd hate to be running critical storage like this on non-redundant hardware and at a minimum the FAS 3170 configured to support this will come in at no less than $100k. Delivering 5x that performance will cost a lot more than 5x the cost of this example. – Helvick Dec 15 '10 at 18:26
And the PAMs are going to help you only with the reads anyway, for the write portion of the task you'd better have some seriously big NVRAMs. – pfo Dec 15 '10 at 18:34
feedback
up vote 1 down vote

Booting 1000 instances of anything concurrently is pretty serious. Even if individual performance is acceptable at 20-30IOPS (which would be a slow disk on a single machine) you're looking at 20-30K IOPS. Get yer checkbook out. It's actually worse than that as most OS's will consume a lot more than is acceptable if there is nothing preventing them. As an example if you have a Windows XP client VM and give it effectively infinite IOPs by connecting it to an SSD array that can deliver 20K or more IOPS I've seen individual virtual machines consume almost 1000 IOPS.

Staging such boot sequences is vital. If these are very low overhead systems then you might be able to get away with about 5 IOPS per system under steady state but the boot storm is called that for a reason. Read\Write IO ratios are also critically important - it's a lot easier (cheaper!) to deliver 5-10K IOPS for read heavy IO but the sustaining IO patterns for typical non server systems are very heavily write biased and it will be much more expensive finding a solution that can reliably deliver 5000IOPs in a 50:50 R/W pattern as it will be to deliver 5000IOPs in an 80:20 R/W pattern.

But seriously - there are very few storage solutions out there that can reliably boot 1000 VM instances concurrently.

link|improve this answer
feedback
up vote 0 down vote

If you have licensed View as a standalone product (not as an add-on to a vSphere cluster), then you will have the full functionality of Enterprise Plus available. From vSphere 4.1 onwards this includes storage resource allocation (at a cluster or resource pool level). Configure and adjust this on your environment to control the IOPs loading from each VM. This will limit and allocate IOPs on a fair voting basis, and also ensure other VMs get a look-in on the SAN IOPs during heavy boot times.

This may extend boot times for the VMs, but will save a fortune on SANs which can deliver such high loadings.

Your other option is to look at the Tiered SANs which include SSD's for the fast tier of storage, such as EMC VNX or the NetApp SANs, but these are not cheap.

link|improve this answer
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.