Virtualisation server for a small business: questions on design, hardware components and vendor [closed]

Question

I need to build or buy a server in order to migrate a small business to its own infrastructure. The business has under 20 employees across 4 sites. Currently, the head office site (where this new server will be located) shares the same IT infrastructure with a sister company.

What the business needs the server to do:

1. Host their low-traffic websites
2. Host their SMTP/IMAP email servers, including and malware/spam filtering
3. Host their Windows file shares
4. Host backups/archives for their 3 satellite sites
5. Not cost too much, say under A$5000

The plan:

1. Buy a single server with good hardware virtualisation capabilities and a reasonable amount of redundancy
2. Setup (at least) two virtual machines:
   • one for any public-facing services (ie. web, email)
   • one to host internal file shares and remote backups
3. Use FOSS software for virtualisation and services, eg. CentOS, KVM/Xen, Apache, PostgreSQL, PHP, Postfix, etc.
4. Do the storage/software setup myself (ie. I don't need the vendor's help with setting up the RAID or OS)

Design considerations:

1. Hardware must be fully compatible with RHEL/CentOS 5.5 x86_64 (eventually the OS will be upgraded to RHEL/CentOS 6). This means no 3rd-party driver disks.
2. Provide all services using virtual machines, for ease of administration (eg. snapshots, rollback)
3. Connect the VM to dedicated LVM volumes on the VM host where applicable (eg. the file server), in order to avoid trapping company data in a VM (think of this as a poor man's SAN/NAS)
4. Don't upgrade this server if more hardware resources are required in the future. Instead, look at building a SAN and perhaps adding a dedicated SAN NIC to this server through which it connects to the SAN

Hardware I'm considering:

• single-socket motherboard with VT-d support and built-in dual-port Gigabit Ethernet
• redudant power supply
• 4-core, Nehalem-based Intel Xeon with Intel VT and IOMMU support
• 12GB (3x4GB) 1333MHz single-rank RDIMMs
• 4x 3.5" drive bays holding 4x SATAII enterprise (ie. decent MTBF/IOPS) drives for full hardware RAID10 (aiming for an approx. 300GB logical drive). SAS would be nice, but it seems too expensive
• slimline DVD-ROM drive
• remote console with dedicated Ethernet port, eg. like DRAC/iLO.
• preferably cross-platform system management software (eg. monitoring, firmware updates, remote console). One vendor wants to sell me a Supermicro-based solution, but this seems to be Windows-only remote console, ie. it runs on Linux too but with only limited features.
• at least two spare PCIe slots for future expansion: one for the Ethernet to a SAN, and the other to a tape drive unit (this is assuming that the sytem or its RAID controller doesn't already have an external jack to attach a tape drive unit to). Note that a tape unit isn't required at this stage as backups will go to a remote over a VPN.

Questions:

1. Can anyone forsee any problems with this setup or have any alternate recommendations?
2. Does anyone already know of a specific server model which accomodates these requirements (either out of the box by some miracle, or with some customisation by the vendor)?
3. Does anyone have any recommendations for a hardware vendor? Please keep in mind that this is a Melbourne-based business. At the moment I am considering Dell and Digicor. I would have included HP, but the boss isn't a huge fan of HP for some reason. The vendor needs to:
   • be reputable
   • be cost-effective
   • have quick turnaround time for replacing faulty parts
   • be able to build custom servers (ie. not just try and sell me some "cookie-cutter" system which doesn't quite meet my needs).
4. Should I just build this myself? This doesn't seem like a popular option among sysadmins, because of the lack of support in case of a breakdown (eg. waiting weeks for the right part to be delivered).
5. Which RAID controller model do I want?
   • how much cache do I need, if any?
   • do I need a battery-backed write controller (BBWC)? I don't think so because it's RAID10, not RAID5/6, but someone may know better
   • is there some open RAID metadata format so that you can switch RAID controller vendors and still have your logical drive work?
6. Is CentOS with KVM/Xen a good idea for the VM host? I can't see why it would be bad, but perhaps there are some horror stories out there.
7. Do I want a dedicated NIC for each VM, or is a software bridge fine?
8. Is there any particular motherboard chipset to get/avoid?
9. Does it cost much more to get the same hardware as above, but a dual-socket board and 8 drive bays? I figure it's better not to get these to keep costs down. If I really need to upgrade, I still can (replace CPU, add RAM, attach to a SAN, etc.)
10. Do I care how many rack units the system takes? I will have enough rack space for anything (eg. 1U to 5U), but is there a benefit to a certain size, eg. cost?

Answer 1

Too little RAM, too little IO.

CHeap end user based:

• AMD board with a 6 crore Phenom II, 16gb Memory. Adaptec 3405 RAID controller and you can fix up 8 discs. I have one of them here, soon two.

• Discs used in this case for me are 4 x wd scorpio black as raid 10, 4x wd scorpio blue as raid 5. the blacks hold all vms and the os, the blue are a large file share... performane is ok, with bottlenecks occasionally.

Or you go long term.

• Amd Opteron based board, two sockets. I have one of those (soon two). 32 to 64 gb ram. Supermicro has a nice cage that has 24 disc bays on a SAS backplance in 2 rack units.... an Adaptec 58005 runs that. Discs are all raptors - so far I hve 12 Velociraptors in there, 300gb each, as RAID 10. 6 for os and virtualization, 6 for a larger database. I started with one processor, 32gb ram. More expensivbe, but high quality and something you can expand on.

12gb ram sound like a bottleneck coming. And the disc layout is going to be problematic. Upragdes to a SAN are stupid - sorry - from a cost point. A SAN makes sense for MANY servers, but to get SAN hardware for ONE server... look at the prices. It is not funny. For 1-2 servers, DAS (Directly attached discs) are mostly a lot better in price.

XEN is a good virtualizer if windows is involved - as XEN is supported natively by WIndows Server 2008.... there are no third party components to install in the virtual machines. Keeps updating a little easier.

For a vendor... SUPERMICRO;) They have all kinds of very nice stuff, especially for high density systems.

Answer 2

Have you taken into account the reliability of your ISP? I would highly recommend hosting web servers and mail servers with a hosting company who can provide the uptime those services need.

You might be able to afford 30 minutes without a file server, but can you afford to have your website down? (Googlebot doesn't sleep!) Or for your email servers to be unable to receive messages? No matter how much hardware redundancy you have, you can't stop the phone/cable company from being stupid.

You should also think about what your ISP will allow. Some block critical ports such as 80 coming in and 25 going out. You can't expect your customers to visit www.yoursite.com:85 because you're forced by your ISP to run your web server on a non-standard port.

Hosting is so dirt cheap you can get a basic "online business card" type website with email receiving for less than $5/mo.

As for file serving, that should be done on-site since disks are cheaper to buy than to rent and there is far less latency over a gigabit lan than 6mbps DSL or Cable.

For mail, our company uses a high-availability (ie. hosted professionally, not on-site) mail server that receives our incoming mail and serves it via POP3s. Then, our on-site server uses Fetchmail to retrieve the messages from the off-site server and places them in Cyrus IMAP for permanent on-site storage and deletes the off-site copy. Our employees access mail via the local Cyrus server.

We do this because long-term IMAP mail storage takes lots of disk space and its cheaper to buy our own disks, but we want the high-availability of being able to receive mail even if the ISP is down.

Are you planning to use Samba to host your files for Windows users? Are you planning to use it as a Primary Domain Controller for central user account management? Is your network mixed with both Windows/Linux clients?

As far as virtual machines, we separate every service into its own machine. This keeps everything isolated in case of any mistakes. That way if someone gets a little overzealous with the rm command and the * for example, you would loose only your mail or only your web server, but not both like you would if they were on the same VM.