Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I need to build or buy a server in order to migrate a small business to its own infrastructure. The business has under 20 employees across 4 sites. Currently, the head office site (where this new server will be located) shares the same IT infrastructure with a sister company.

What the business needs the server to do:

  1. Host their low-traffic websites
  2. Host their SMTP/IMAP email servers, including and malware/spam filtering
  3. Host their Windows file shares
  4. Host backups/archives for their 3 satellite sites
  5. Not cost too much, say under A$5000

The plan:

  1. Buy a single server with good hardware virtualisation capabilities and a reasonable amount of redundancy
  2. Setup (at least) two virtual machines:
    • one for any public-facing services (ie. web, email)
    • one to host internal file shares and remote backups
  3. Use FOSS software for virtualisation and services, eg. CentOS, KVM/Xen, Apache, PostgreSQL, PHP, Postfix, etc.
  4. Do the storage/software setup myself (ie. I don't need the vendor's help with setting up the RAID or OS)

Design considerations:

  1. Hardware must be fully compatible with RHEL/CentOS 5.5 x86_64 (eventually the OS will be upgraded to RHEL/CentOS 6). This means no 3rd-party driver disks.
  2. Provide all services using virtual machines, for ease of administration (eg. snapshots, rollback)
  3. Connect the VM to dedicated LVM volumes on the VM host where applicable (eg. the file server), in order to avoid trapping company data in a VM (think of this as a poor man's SAN/NAS)
  4. Don't upgrade this server if more hardware resources are required in the future. Instead, look at building a SAN and perhaps adding a dedicated SAN NIC to this server through which it connects to the SAN

Hardware I'm considering:

  • single-socket motherboard with VT-d support and built-in dual-port Gigabit Ethernet
  • redudant power supply
  • 4-core, Nehalem-based Intel Xeon with Intel VT and IOMMU support
  • 12GB (3x4GB) 1333MHz single-rank RDIMMs
  • 4x 3.5" drive bays holding 4x SATAII enterprise (ie. decent MTBF/IOPS) drives for full hardware RAID10 (aiming for an approx. 300GB logical drive). SAS would be nice, but it seems too expensive
  • slimline DVD-ROM drive
  • remote console with dedicated Ethernet port, eg. like DRAC/iLO.
  • preferably cross-platform system management software (eg. monitoring, firmware updates, remote console). One vendor wants to sell me a Supermicro-based solution, but this seems to be Windows-only remote console, ie. it runs on Linux too but with only limited features.
  • at least two spare PCIe slots for future expansion: one for the Ethernet to a SAN, and the other to a tape drive unit (this is assuming that the sytem or its RAID controller doesn't already have an external jack to attach a tape drive unit to). Note that a tape unit isn't required at this stage as backups will go to a remote over a VPN.

Questions:

  1. Can anyone forsee any problems with this setup or have any alternate recommendations?
  2. Does anyone already know of a specific server model which accomodates these requirements (either out of the box by some miracle, or with some customisation by the vendor)?
  3. Does anyone have any recommendations for a hardware vendor? Please keep in mind that this is a Melbourne-based business. At the moment I am considering Dell and Digicor. I would have included HP, but the boss isn't a huge fan of HP for some reason. The vendor needs to:
    • be reputable
    • be cost-effective
    • have quick turnaround time for replacing faulty parts
    • be able to build custom servers (ie. not just try and sell me some "cookie-cutter" system which doesn't quite meet my needs).
  4. Should I just build this myself? This doesn't seem like a popular option among sysadmins, because of the lack of support in case of a breakdown (eg. waiting weeks for the right part to be delivered).
  5. Which RAID controller model do I want?
    • how much cache do I need, if any?
    • do I need a battery-backed write controller (BBWC)? I don't think so because it's RAID10, not RAID5/6, but someone may know better
    • is there some open RAID metadata format so that you can switch RAID controller vendors and still have your logical drive work?
  6. Is CentOS with KVM/Xen a good idea for the VM host? I can't see why it would be bad, but perhaps there are some horror stories out there.
  7. Do I want a dedicated NIC for each VM, or is a software bridge fine?
  8. Is there any particular motherboard chipset to get/avoid?
  9. Does it cost much more to get the same hardware as above, but a dual-socket board and 8 drive bays? I figure it's better not to get these to keep costs down. If I really need to upgrade, I still can (replace CPU, add RAM, attach to a SAN, etc.)
  10. Do I care how many rack units the system takes? I will have enough rack space for anything (eg. 1U to 5U), but is there a benefit to a certain size, eg. cost?
share|improve this question

closed as not a real question by Chris S Jan 20 '13 at 14:16

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.If this question can be reworded to fit the rules in the help center, please edit the question.

    
I'm wondering if it is realistic to expect someone to be able to completely answer this question. I was thinking that I should have posted it as separate SF questions, but I opted for an all-in-one so that the reader would have a reasonable level of context with regard to the situation. –  mosno Nov 4 '10 at 0:37

2 Answers 2

up vote 2 down vote accepted

Too little RAM, too little IO.

CHeap end user based:

  • AMD board with a 6 crore Phenom II, 16gb Memory. Adaptec 3405 RAID controller and you can fix up 8 discs. I have one of them here, soon two.

  • Discs used in this case for me are 4 x wd scorpio black as raid 10, 4x wd scorpio blue as raid 5. the blacks hold all vms and the os, the blue are a large file share... performane is ok, with bottlenecks occasionally.

Or you go long term.

  • Amd Opteron based board, two sockets. I have one of those (soon two). 32 to 64 gb ram. Supermicro has a nice cage that has 24 disc bays on a SAS backplance in 2 rack units.... an Adaptec 58005 runs that. Discs are all raptors - so far I hve 12 Velociraptors in there, 300gb each, as RAID 10. 6 for os and virtualization, 6 for a larger database. I started with one processor, 32gb ram. More expensivbe, but high quality and something you can expand on.

12gb ram sound like a bottleneck coming. And the disc layout is going to be problematic. Upragdes to a SAN are stupid - sorry - from a cost point. A SAN makes sense for MANY servers, but to get SAN hardware for ONE server... look at the prices. It is not funny. For 1-2 servers, DAS (Directly attached discs) are mostly a lot better in price.

XEN is a good virtualizer if windows is involved - as XEN is supported natively by WIndows Server 2008.... there are no third party components to install in the virtual machines. Keeps updating a little easier.

For a vendor... SUPERMICRO;) They have all kinds of very nice stuff, especially for high density systems.

share|improve this answer
    
Thanks, TomTom, I will take a look at your recommendations. I would love to get SAS for high IOPS but it seems a little expensive? I only mentioned SAN because that is likely the future direction for this setup in terms of adding redundancy (more servers, with live migration of VMs). In the meantime, I can look at ensuring that the case/backplane has spare bays for extra DAS drives. I am curious why you say the RAM is a bottleneck, as I can always add more to this server (which is part of the reason I chose RDIMMs as opposed to UDIMMs). [ps. sorry for the late response as I have been on leave] –  mosno Nov 4 '10 at 0:24
    
Well TomTom, the boss has opted that we just get a basic desktop-class server up and running for now. I have just purchased parts for a Phenom II X4-based system: Phenom II 955 X4 Black Edition processor, 2x4GB Kingston PC3-10600 DDR3 RAM, Gigabyte GA-880GM-USB3 motherboard, 4x WD Caviar RE3 500GB SATA II HDDs, Antec BP500U Basiq 500W ATX power supply, CoolerMaster CM 690 case. I am going to accept your answer since a) my question was probably too complex as previously mentioned, and b) you reminded me that AMD-based systems can be cost-effective. Thanks! –  mosno Nov 8 '10 at 4:03
    
ps. 1) I know the RAM is low, but the board can take another 8GB if need be. 2) I chose the RE3 HDDs because they are apparently designed for high load use. 3) The boss said if we need a faster system, he is happy for me to convert this one into a desktop and then buy a proper server. –  mosno Nov 8 '10 at 4:05

Have you taken into account the reliability of your ISP? I would highly recommend hosting web servers and mail servers with a hosting company who can provide the uptime those services need.

You might be able to afford 30 minutes without a file server, but can you afford to have your website down? (Googlebot doesn't sleep!) Or for your email servers to be unable to receive messages? No matter how much hardware redundancy you have, you can't stop the phone/cable company from being stupid.

You should also think about what your ISP will allow. Some block critical ports such as 80 coming in and 25 going out. You can't expect your customers to visit www.yoursite.com:85 because you're forced by your ISP to run your web server on a non-standard port.

Hosting is so dirt cheap you can get a basic "online business card" type website with email receiving for less than $5/mo.

As for file serving, that should be done on-site since disks are cheaper to buy than to rent and there is far less latency over a gigabit lan than 6mbps DSL or Cable.

For mail, our company uses a high-availability (ie. hosted professionally, not on-site) mail server that receives our incoming mail and serves it via POP3s. Then, our on-site server uses Fetchmail to retrieve the messages from the off-site server and places them in Cyrus IMAP for permanent on-site storage and deletes the off-site copy. Our employees access mail via the local Cyrus server.

We do this because long-term IMAP mail storage takes lots of disk space and its cheaper to buy our own disks, but we want the high-availability of being able to receive mail even if the ISP is down.

Are you planning to use Samba to host your files for Windows users? Are you planning to use it as a Primary Domain Controller for central user account management? Is your network mixed with both Windows/Linux clients?

As far as virtual machines, we separate every service into its own machine. This keeps everything isolated in case of any mistakes. That way if someone gets a little overzealous with the rm command and the * for example, you would loose only your mail or only your web server, but not both like you would if they were on the same VM.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.