I want to finally spend a little money upgrading my company's virtualization infrastructure, but I've been looking at hardware for a couple of weeks now and I'm not quite sure what's appropriate for our needs. In addition to buying some actual server hardware, I want to replace our current VMware Server with VMware ESXi. The initial budget is probably somewhere around $2000 but may be able to eventually scale up to $3000 if we need to tweak the performance later.
After reading some scary articles on RAM errors and hard drive failure rates over on StorageReview (and after dealing with several disk failures myself), I want to move to ECC RAM and I want to store our VMs on redundant ZFS volumes to protect against silent data corruption and data loss.
Note: I know there are already various questions about VMware ESXi and ZFS hardware recommendations, but I couldn't find any discussions about the performance of VMware ESXi with the VMs stored on ZFS.
- VMware Server on Windows 2003
- Athlon X2
- 8 GB RAM
- 2x250 GB SATA mirror (dynamic disk)
- 2x640 GB SATA mirror (dynamic disk)
Right now we run 4 VMs 24/7 on an Athlon X2 with 8 GB RAM. Occasionally we need to fire up an additional VM or two, and the performance suffers. We also want to virtualize a couple additional servers, but I don't see us needing more than a dozen anytime in the next couple of years. I've read that VMware ESXi scales better since it's a bare-metal hypervisor and it has various performance optimizations.
- 2 GB RAM, Win2003 (running Fogbugz, including MS SQL Server)
- 512 MB RAM, Fedora Linux (running legacy systems such as MediaWiki, RT, and Bugzilla)
- 2 GB RAM, WinXP (regularly used for software development & testing)
- 1 GB RAM, WinXP 64-bit (only occasionally used for testing)
- 1 GB RAM, WinXP Japanese (only occasionally used for testing)
- 1 GB RAM, CentOS Linux (regularly used for software development & testing)
- Entry-level Dell or HP quad-core Xeon server
- Dual quad-core Xeon 55xx (one CPU initially--is Nehalem worth the extra money?)
- 8-16 GB ECC RAM (max capacity of at least 32 GB RAM)
I figure that will set me back $1000-$1500 if I buy the RAM separately from NewEgg, but I'm not sure if it's better to put more money toward a newer-generation Xeon, if I should put it toward a second CPU, or if I should just save it for now.
- 1.5 GB RAM, Windows 2003 Small Business Server (currently only used as the domain controller; will probably be rebuilt/replaced soon)
- 2 GB RAM, Windows 2003 Server (infrequently-used Terminal Server, along with various license servers for different parts of our development environment)
- A few other Windows and Linux workstation VMs used occasionally for testing
I figure I could repurpose the current Athlon X2 server as the ZFS server by installing one of the free Solaris derivatives. (Thanks MarkM for pointing out that the OpenSolaris group disbanded after Oracle discontinued their support a couple weeks ago--I'll have to follow the progress of Illumos' open-source fork). I'm planning to make the storage available to ESXi via iSCSI over a direct gigabit Ethernet link.
The motherboard supports 8 SATA drives, and I figured I would add an additional 4- or 8-drive eSATA enclosure so I can have several ZFS mirrors or raidz2 volumes.
I'll probably replace the memory with ECC RAM for data integrity, and might add SSDs for the ZIL/L2ARC to improve ZFS performance, but I'm not sure if the money spent on SSDs would be better spent on SAS drives instead.
Given these specs, am I sizing the servers correctly? Will a mix of SSD and consumer/"enterprise"-grade 7200 RPN SATA drives give us sufficient disk performance, or do we really, really need to upgrade to SAS? Are there any gotchas I need to watch out for? Am I missing something important?
Most importantly, has anyone had experience setting up a similar configuration--and, if so, how did it perform?