4

Has anyone done, or is aware of any, calculations to compare the cost of power and cooling compared to the cost of hardware (servers) in a typical data center? This is to compute a true total cost of ownership of self-hosting servers. Of course real TCO includes: hardware_cost + power + cooling + rental + human_cost + maintenance

Is there any study that says something like (TCO - hardware_cost) = 40% of hardware_cost in 3 years?

Any pointers will be appreciated.

7 Answers 7

3

It's going to vary by area, with what your costs for power are. This varies greatly, and tends to be why Google, Microsoft and others are building sites in what might otherwise be considered the middle of nowhere.

... and you missed the most overlooked item in TCO : disposal (transfer or destruction of the data; physical removal and sale/recycling of the hardware).

3

What you really want to know about is PUE.... a measurement of power/cooling use vs Server power use.

Power to Useful Equipment (PUE)

PUE used to rate data center designs, and for traditional hardware you get get approximately 2.6

which means that for every watt of power going to your server, 1.6 watts get used on plant and cooling.

Now.... taking some really typical stuff, a Dell PowerEdge R710 ( their green model) will use 90-120 watts at idle. At 100% CPU its more like 230 watts. It costs $1299 US.

Now at 0% CPU that will total up as 100W from the Dell, and another 160W from plant. So you spend 260W, 24x7 for no computing.

assuming power at 10c per kw/hr ( more pricey in say, california, more like 15c there) that is $4.36 per week; $227 per year; And that's at idle!

Now at 100% cpu, the total load is about 680W. Most of that load is inefficiency in the infrastructure and air cooling. That cost you $11.42 per week, or $594.00 per year.

So in two just over two years, at 100% CPU the server would cost more in electricity than it cost to purchase.

At which point a new server (same $) would have more than twice the processing power for the same power consumption

It's a really good idea to replace equipment more often than you might think, to get back to more efficient hardware.

Where I work, we do offer an energy consultancy to data centers.

Free advice: virtual machines cost less. Compress idle-ish machines into VM servers,

Get a VM server that will automatically turn on and turn off physical servers and migrate VM's around as load changes.

This can save a fortune!

People spec system much better than PUE of 2.6. A PUE of 1.2 is quite achievable, but is not a plain old industrial building with racks of servers and big air conditioning units.

The really picky people start looking at tPUE, where you consider power lost in the PC PSU , CPU voltage regulators and cooling fans as "wasted" instead of useful.

James Hamilton is a big blogger in this space, he has a great article over at

http://perspectives.mvdirona.com/2009/06/15/PUEAndTotalPowerUsageEfficiencyTPUE.aspx

8
  • 2.6? My data center just put something on my table that they certified with a PUE of 1.39.
    – TomTom
    Nov 11, 2010 at 0:48
  • You need 2.6 times the power used to cool a server? Man, your servers are either Prescott-based or have poor airflow, or your cooling system is horribly inefficient; or all three. Nov 11, 2010 at 3:22
  • To be accurate, a PUE of 2.6 means for every Watt going into your server, 1.6 Watt go into infrastructure and cooling. See for example google.com/corporate/green/datacenters/measuring.html Nov 11, 2010 at 14:58
  • A PUE of 2.6, sadly, is high, but not a total outlier. And what type of hardware you use for your servers is only a fraction of the story. There's plenty of stuff that you can do about how you cool the servers, and it makes a huge difference. I once listened to a talk on an IT conference where one company explained how they ran their datacenter at a room temperature of 40° C with commodity hardware and a quite unorthodox, but very professional approach to cooling. Nov 11, 2010 at 15:04
  • 1
    Guys, I used 2.6 in my example because that's about what a "datacenter" put together out of "the usual" pieces achieves. Take one industrial/office building, (make it energy inefficient; it will be) add big air conditioning (don't use an economizer, or ambient air), have lots of fluorescent lighting (which is left on), set the thermostat for 20C machine space, and you're looking at 2.6ish just like that. The last place we measured got 2.3 and is probably one of the best ones in town. If you even know what PUE is, you probably know that 1.2-1.3 is possible. Most people don't. Nov 11, 2010 at 21:57
1

APC as well as the actual server vendors (IBM, Dell, HP) all have calculators to come up with how much it costs to "go green" so to say.

For example: APC's data center efficiency portal has a lot of good information. You have to register first, but it is worthwhile to get access to the calculators, etc.

http://www.apc.com/tools/registration/promo/RegisterCustomer.cfc?method=getPromo&keycode=c828w&promotionID=13186

0

I've seen anecdotes about power consumption (possibly including cooling) for servers can now exceed the cost of the hardware in just a couple of years, but no hard numbers. As noted by Joe H, this will vary by area - if you're in California and someone starts playing with your energy markets, you may find yourself paying more than if you're next to a wind farm in Wyoming.

Reducing these costs is a big part of the push to virtualization, so look into some of the information available there as well. I suspect that VMWare, Citrix and possibly MS have white papers and marketing fluff relating to the power & cooling savings from virtualizing servers.

0

Its going to be really difficult to get a comprehensive & accurate answer without specifically studying your environment. Examples:

  • When you walk around in your server room/datacenter, are you cold? If the answer is yes, you're wasting money.
  • How many servers do you have? What is your growth rate in computing demand? What is the average utilization of these servers? If you can't answer that, you are probably overbuying.

Joe H. was right on the money regarding disposal. Equipment should have a budgeted lifecycle, from acquisition to decommissioning. If you think about the entire lifecycle of IT equipment (including desktops) when you buy it, you will naturally start slashing away at operational inefficiencies.

0

Cooling actually splits into ventilation and cooling, which both also split into hardware and power. Also, ventilation and cooling are to some degree interchangeable (you can move warmer air faster to get the same amount of heat transported away), which means you can reduce cooling demand by cranking up the air flow.

I know, this doesn't exactly make your task easier, but it might still help to get the numbers sorted out. Cooling + ventilation power can easily be the same as server power, reducing that was the chief goal behind Google's efforts in their data centers.

I'm not too aware of the prices of cooling and ventilation hardware, but from the odd number I have heard along the way, I'd guess that these, too, might well be in the same ballpark as the servers.

Generally speaking, I think it's fair to say that the cooling strategy is at least 50% of a TCO-aware planning.

4
  • The same? Crap center. Mine just certifieid wih 1.39 ;)
    – TomTom
    Nov 11, 2010 at 0:48
  • @TomTom - Having a PUE of 1.39 sadly isn't even close to common. 2 and more still seems the rule. If there's a way to share how your datacenter operates in that regard, this could probably serve as a guidance for many others. Nov 11, 2010 at 14:51
  • Ouch. Really ouch. Stinks. Seriously. Especially as the 1.39 center still uses not needed active cooling. How crappy are some "pfofessional" centers?
    – TomTom
    Nov 11, 2010 at 14:56
  • @TomTom - Well, according to numbers cited by Google (google.com/corporate/green/datacenters/measuring.html), 2 is typical, 1.7 is "improved operations", 1.3 is "Best practices" and 1.2 is "State-of-the-art". Google says they run at 1.18. Nov 11, 2010 at 15:33
0

Current estimates are that the average PUE of a medium Data Centre is 2.6. So as others have said this means for every 100kW of IT you have 160kW going into the cooling & other losses. However to get 100kW of cooling doesn't need 100kW of electricity. The factor between power in & cooling capacity is known as COP or Co-efficient of Performance & for a typical modern chiller will be 3.5-4 so at COP 4 you'd get 100kW of cooling for 25kW of electrical energy. However a lot of Data centres waste huge amount of cold air with recirculation of hot & cold air because of poor design, lack of blanking panels etc.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .