Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We are going to build a Large Enterprise Cloud Solution and would like to know in which configuration we would have a better redundancy/performance and cost efficient.

Here are 2 examples which we're considering as good configurations to grow.

Main Hardware:

  • 42U Rack
  • 2x Dell Force10 S4810
  • 2x HP P2000 iSCSi (2x AW595A Controller), 12x 600GB SAS (RAID10)

Example 1 for a configuration with many small nodes:

  • 36x HP DL120 G7 (Xeon® E3-1240 @ 3.30GHz), 32GB RAM, 2x10GE, 2x240GB SSD (RAID1)

Example 2 for a configuration with fewer, bigger nodes:

  • 18x HP DL380p G8 (2 x Xeon® E5-2640 @ 2.5GHz), 256GB RAM, 2x10GE/4x10GE, 2x240GB SSD (RAID1)

So the question is now, is it better to take 36x DL120 or 18x DL380?

Would the DL380 handle for example 40GE (2xNIC) or is that to much to handle for the server? Even if we would set up 40GE, is the RAM just too large for the DL120, or is the 32GB RAM not enough resources and won't be cost efficient because of the expensive switches and iSCSi devices?

How do other companies operate, like Rackspace, Amazon, Facebook, Google etc. (I mean they have a very large number of servers, I know, but for me the main strategy is what is important to know.)

share|improve this question

closed as not constructive by Sven, ewwhite, Scott Pack, mdpc, Michael Hampton Dec 26 '12 at 4:17

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

I don't think the SAN choice is a good fit for what you're doing... – ewwhite Dec 25 '12 at 14:21
Why you go with 10G? I am asking because Infiniband is faster, lower latency AND cheaper. By a LOT (we talk of 40 gigabit right now being standard). – TomTom Dec 25 '12 at 14:25
Yea i was thinking the same but i wasn't sure if for example the DL120's can handle such throughput. And to run everything under 40GE would require a Force10 Z9000 Switch (100 grand each). I thought maybe 2x 10GE Cards (2x2x10GE) would be the same but as i said already, i don't know if the nodes will work with it. – Baloo Dec 25 '12 at 17:03
One more Question: How much RAM i can handle MAX. (with each node) before the network throughput will be full? For example the InfiniBand are used in DL580 G7 and i would love to work with them but if i put there 512GB RAM and the 4x CPU's, isn't it to much RAM/CPU for just 40GE? – Baloo Dec 25 '12 at 17:13
up vote 0 down vote accepted

Your approach will depend on the actual software you want to run on the cloud.

For example some server implementations can not be split among several nodes and would benefit a single machine with more cores. About the 10GE or 40GE NIC's: Do you require lots of I/O? If you run some kind of computation heavy algorithm you won't benefit from more storage or networking throughput.

Bottom line: If you don't supply any information about the actual software you are going to run on the cloud nobody can tell you which configuration will be the best.

share|improve this answer
Hey, we are going to run with ESX (VMWare vCenter Orchestrator) and probably Cloudstack. We need an high I/O and large computation power for example Video Convertation. Thanks, Baloo – Baloo Dec 25 '12 at 12:37
If the sole purpose of the cloud is video-encoding, using ESX quite expensive and unneeded. It will have both a performance and cost-wise hit which is totally unnecessary. Use bare servers to run video encoding. If you download the videos from the SAN to the SSDs, then encode the videos and afterwards upload them back to the SAN, the DL120 will have better performance (because the total SSD throughput of all the DL120s combined is higher) – Tim Dec 25 '12 at 13:16
@Tim exactly. There is little reason to use virtualization in a HPC setup. Mostly because you run identical nodes anyway. I build up something similar for math calculation - everything is Hyper-V based, EXCEPT the computing nodes. – TomTom Dec 25 '12 at 14:27
Besides of the Video Encoding, we want to rent out like "Private Clouds" to our customers so we need kinda a All-in-One Solution. ESX was for now just used to rent VDS to our Customers but we have many customers who desire a larger Solution then just what the DL120 got (32GB RAM and 4 cores). As well is the ESX Solution in Combination with the vSphere Client really easy to manage and would be nice to set up the Cloud so that it fit perfect to it :) – Baloo Dec 25 '12 at 16:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.