4

Im looking into the cost of putting a Cassandra cluster into a colo facility. Along these lines there would be 6-8 servers at the outset with expected growth over time. One option is just a series of Dell R320 (or similar). Another option would be blades or similarly built machines that share power.

Looking at the details of an 8 node system I see it has 4x1620 watt power supplies. This gives a total of 6480 watts. If I have a rack with 208V this means I'm pulling more than 30A at peak. So I've maxed out my 42U rack in 6U of space. I realize this is 'peak load' but it seems a bit extreme.

Am I misunderstanding how this calculation works? I get VA=W and I get that it won't pull this kind of load but 30A is a lot of current. I don't have the luxury of buying one and using a kill-a-watt to accurately measure it. The specs for the system don't make it sound like these are redundant but that's a tremendous amount of current.

Has anyone deployed blades or multi-node servers and measured the required current? I'd love to get a Dell M1000 but the prospect of trying to budget for 40A just makes me need to lie down.

EDIT If I use a kill-a-watt to measure the input current for a system with n power supplies - do I sum them? Are they all pulling 1/n?

2
  • 3
    VA is not quite watts. May 18, 2013 at 4:28
  • 3
    Welccome to the world of low end data centers. Ask them for a rack that can handle 15, 20 or 30kw. This is possible - but it will COST. Most hosting centers just are not prepared for high density systems. I ahve seen power budgets that qould require atoms to allow more than 1 processor per 2RU. Btw., you likely have no 208V - you should have 230. That was standardized long long long time ago to 230.
    – TomTom
    May 18, 2013 at 6:35

2 Answers 2

8

Yes, blades are dense. :)

You need to use a power budgeting tool to determine maximum power draw of your particular hardware configuration. Your reseller should be helping you with this. (since that's what I do :)

Multiple power supplies can have quite a few possible scenarios:

  • N+N configuration: Maximum power draw of N×Wattage power.
  • N+1 configuration: Maximum power draw of N×Wattage power.
  • N, non-redundant: Maximum power draw of N×Wattage power.
  • N+X configuration, throttling allowed: Maximum power draw of N×Wattage + C

Your configuration of 4×1620W is probably N+N, so maximum draw is around 3240W plus a bit. But check the documentation! It's also likely that each of the above scenarios is software-configurable, take note of that.

Oh and by the way, VA=W×Power Factor.

3
  • On top, simply said: they ARE dense. Normal racks just can not hanle the cooling - that needs actively cooled racks with their own internal heat exchange. It is not a problem to get 30kw into a rack, but it will COST. Hosting centers in general are not prepared for high end machines - their power density partially is a low end joke.
    – TomTom
    May 18, 2013 at 6:34
  • Normal racks are perfectly capable of handling cooling requirements for a rack full of blades, you only need to deliver sufficient air. The last DC I was in had 40U of blades per rack (2x 377V 60A 3Ø supply) with sealed minimal cooling aisles. Works great when you do it right. But as soon as you try to go into a random hosting provider who gives you 120V 15A for 42U, not bloody likely.
    – MikeyB
    May 19, 2013 at 13:03
  • yeah, but doing it right is hard - much easier to get a closed cooled rack. Problem is - mostly what I wanted to say - the environment must be prepared for those densities. And hosting providers just are not.
    – TomTom
    May 19, 2013 at 16:24
4

Just to note, the math is off, too.

I see it has 4x1620 watt power supplies

Good.

If I have a rack with 208V this means I'm pulling more than 30A at peak

Yes, using 1 phase.

You are off in three things:

  • One, it likely is 230V, not 208.
  • Two, that is no 4x1620Watt - it is 3x1620W + 1 spare power supply.That smells like a 3+1 configuration - that is quite important. How much power can that cabinet pull?
  • Three, you would use a 3 phase setup here, no 1 phase, and that REALLY cuts down on your amps. By a factor of 1.73 to be exact.

THAT SAID: yes, blades are dense and yes, you need special racks for that and a lot more power density than a normal cheapo colocation center is willing to handle.

This is not about your power - directly. It is about your heat. When the data center is planned, they simply do not foresee people blowing 20kW in a rack, so the cooling is not there. There are special racks for up tio IIRC 30kW in a rack, but they COST and basically have their own internal air conditioning, so the heat goes out in a fluid not into the air. This will COST - and brutally speaking most data centers I have seen are not prepared for that at all.

Especially webside colocating is - ah - sometimes SO low power in their racks it is a joke and we can only hope Intel really gets the power down on computers. I can normally not fill a rack with 2 processors per unit and expect the data center to handle that, and dual socket 1u machiens are not exactly high density.

I'd love to get a Dell M1000 but the prospect of trying to budget for 40A just makes me need to lie down.

Besides the fact that I could never make sense out of this Dell thing financially - SUperMicro has cases with 2 computers / RU (check their twin fat cases) that cost a LOT less and do not require super expensive additional stuff - this IS costly. First, you will pull a LOT of power, which means a lot of additional cost and investment for cooling. Second, the power bill just will be high. Live with it. Hope for less power hungry chips ;)

5
  • The 8 node system in question is from SuperMicro. It doesn't say that it's 3+1 but that's probably a good guess. I'll go back and read the specs again.
    – ethrbunny
    May 18, 2013 at 11:38
  • @ethrbunny If that is a twat fin it fully redundant - not 3+1 but 2x(1+1) - i.e. 2 power supplies for the lower 4, 2 for the upper 4, and that redundant (i.e. lower has 1+1, upper has 1+1).
    – TomTom
    May 20, 2013 at 8:39
  • Um sorry - 'twat fin'? I'm not familiar with this term. Can you provide any more detail?
    – ethrbunny
    May 20, 2013 at 15:03
  • fat twin - go to the SuperMicro website. It is a small case, 4 RU, for 4 small computers (special form factor). 8 node is either a non-full SuperMicro Blade, or a "fat twin", which is their term for a special form factor.
    – TomTom
    May 20, 2013 at 15:20
  • Actually 208V is quite common in data centres.
    – hookenz
    May 22, 2013 at 22:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .