21
votes

For the purpose of this question, I am interested in server/datacenter related hardware.

Have you had any measureable amount of ROI by swapping existing hardware to more "green" or energy efficient hardware?

For example, VMWare says you can reduce energy consumption by up to 80% by using virtualization. I have also heard of a cooling solution from HP which is suppose to reduce a small amount of engery usage (<25% I think). Google has also done something by integrating a UPS into their power supplies to reduce energy consumption.

Any real world experiences would be great, but if you have any details on initial cost, savings and pay off time about what changes were make that would fantastic. I am not only interested in virtualization, I am interested in anything.

3
  • Great idea! We're wrapping up a small p2v consolidation, and including some ROI/$$ on the after-action report will go a long way toward building support for my next proposal. Jun 5, 2009 at 14:07
  • Go support my site proposal at area51.stackexchange.com/proposals/15640/… to create a great site for all energy efficiency questions. Aug 20, 2010 at 15:26
  • This question is off-topic under current topicality rules. Jan 22, 2015 at 5:27

16 Answers 16

11
votes
+100

This is a pretty huge question. There are alot of different ways people are saving energy and then in turn saving $$ by greening their datacenters.

I think you are asking about the ROI on such efforts.

Virtualization is certainly a great example - purchase some big physical servers (2 or 4 sockets) with a ton of memory (32 - 128 GB) and then convert Physical Servers to Virtual Server (P2V) and turn off the physical servers.

VMware has an ROI caculator

http://vmware.com/go/calculator

But it really depends - if you put in 3 VMware ESX Hosts that can handle 40 Server Workloads and you only migrate 10 or 20 - then you won't see the ROI.

But if you counted the 40 power supplies (2 per server or 900 Watts per box) and then realize you are replacing them with 3 ESX hosts (2 per server or 1200 watts per box) then you end up yanking out 36,000 Watts (40 x 900) and replacing it with 3600 watts of power (nearly 10:1 consolidation from a Watts perspective). A net savings of around 32,000 Watts.

And realize - its approximately 1 Ton of Cooling per every 3.5 kW of power - so if you take out 32,000 Watts then you can save approximately 30 tons of cooling (so you figure the cost of power to 30 tons of cooling) and that's a savings you can measure.

My math be off - I been playing outside all day (sun exposure).

Other big areas to improve in - Distributed Power Management - turn off or condense workloads at night, spin down disks in a SAN if not needed, etc.

Replace batteries in UPS systems with Flywheels (http://www.pentadyne.com/).

Use ambient cooling if possible, when possible.

Put your Datacenters somewhere with a cheaper cost per kW/hour.

Use Deep Lake Water Cooling (shout out to Toronto peeps).

2
  • ROI was the acronym missing from my vocab today, thanks, I will update my question. I haven't come across flywheels yet, that sounds very interesting, thanks!
    – Bob
    May 31, 2009 at 23:04
  • UPS batteries are constantly charging and discharging so they almost always are drawing power - a flywheel or a couple fly wheels store enough kinetic energy upon discharge to provide 3-5 minutes of power (or enough time for a generator to kick in) and then they reset in case of a another power outage. The drawback may be you need a couple of them.
    – Rob Bergin
    May 31, 2009 at 23:31
6
votes

As everyone has commented on VMware, I'll talk about something different.

Another way we reduce energy consumption is to not run old hardware. The servers we bought over 5 years ago consume more than twice the power, and output nearly twice the heat of low energy servers today.

Tips:

Look for the BTU ratings of servers before you buy, try and use efficient power supplies and low thermal-power CPU's.

Decommission servers fully once they are no longer needed (i.e: remove them from the data center entirely)

/ Richy

5
votes

Microsoft has a really outstanding whitepaper on server power management:

Power In, Dollars Out: How to Stem the Flow in the Data Center

I learned a lot from reading it, and was particularly surprised by these idle power server consumption graphs.

Server A is of 2005 vintage:

Server B is of 2008 vintage:

Now these are fairly beefy servers, with 64 GB and 128 GB of memory respectively. Still, that does not bode well for the future, since memory is the #1 thing that most servers need a ton of and are getting more of every year.. yet unlike the latest-n-greatest CPUs it has no idle throttle and sucks down tons of power 100% of the time!

Another interesting stat:

according to the United States Environmental Protection Agency (EPA) “Report to Congress on Server and Data Center Energy Efficiency,” (PDF link) production servers run at anywhere from 5 to 15 percent utilization on average.

4
votes

As everyone has noted, there are lots of parameters to consider.

The most basic one is power. Check with your datacenter on the difference in rates for them providing 220V vs 110V power - you might be suprised. Do they offer an 18V DC option? (this is most often available in datacenters that cater more to telecom gear, as that's been the standard there for decades) Power supplies exist to let x86 gear run off of all the above. IIRc, the DC power supplies are most efficient, followed by the 220V and then the 100V. DC is most efficient because it's just a voltage drop via resistor, not an AC/DC converter as is in AC power supplies (the outputs of a PC power supply are +/-12V and +/-5V, right?). Sometimes it's even most cost effective to put a per-rack AC/DC converter and then run the AC to the converter in the rack and then short-run the DC power up to the machines.

Besides type of power, there's plain old power use: spinning down unused drive arrays, setting up unused machines to sleep in a low-power mode, or to come online when needed (via remote power control or wake-on-lan) but othewise shut themselves down, virtualization to consolidate workloads into fewer physical machines while maintaining logical separation.

The effects of power consumption trickle all through the equation: less power means less waste heat means less cooling necessary. It's the cornerstone of going green... but it's also complicated and can be difficult to measure.

My favorite way to be green these days is actually to punt and 'use the cloud'. Amazon and Google and Rackspace and Linode are better at this than me, and probably better at it than you too. Figure out how to apply the cloud to your problem, thereby making the entire datacenter-provisioning problem... someone else's.

2
votes

VMware can get you some very nice power savings. By way of example I had a site with 8 servers that were only used during the day, these were on 8 1RU boxes running a single threaded CPU bound application. I moved them to 2x 1RU Quad Core Machines and enabled DPM.

At night and during certain periods all machines were migrated to 1 server and the second server was shut down. So from 8 servers down to 1 (plus SAN) for a significant time.

If the power saving alone will ever pay for VMware licenses is a different question.

DPM in my mind will be the nice visible cost saver though. Depending on your environment of course certain hours require significantly less CPU time than others and the ability to save power and maybe wear and tear during that time will save you money.

2
votes

Here's a good article on virtualization and going green, and the use of Solaris Zones compared to VMWare (disclaimer: I work for the guy who wrote it).

2
votes

We have a Nexsan SATABeast as primary storage for our SAN - it can automatically park heads (~20% power savings), spin down the SATA-disks to 4000rpm (40% power savings) and eventually shut the disk downs (yep, 100% savings). The feature is called AutoMAID.

I have to admit that I'm not using this feature on the arrays that host the Vmware OS-VMFS volumes, as I dont want them to lag. It's possible to just select head parking and spindown, and always let the disks run.

Another thing we've done is to run a heat inverter (instead of A/C) in the server room. It's like Air Conditioning, except that it's two-way airflow. It keeps the server room around 20C, and pushes the hot air down to our basement (where it heats up the basement and keeps our floors in the 1st floor a bit warmer).

2
votes

Here's a quick win we implemented: enabled double-sided printing. Most modern printers can easily handle this, and it saves on energy, ink, and paper.

1
  • 1
    umm, wouldn't it just save on paper?
    – Antitribu
    Nov 26, 2009 at 17:10
1
vote

VMWare isn't the only way to save energy and dollars in a data center. You need to think of your compute resources as an single system and tweak/refactor to improve the systems performance and efficiency. That's the difference between engineering and goofing around with computers.

Some examples:

  • Cooling: lots of places with raised floors place perforated tile in "hot aisles". That's like running a space heater in your freezer.
  • Cooling: designing datacenter space to be comfortable for humans
  • Energy: Using old equipment to provide services because it has been depreciated and is "free"

There's no magic number that makes a particular scheme for saving energy or other resources right for you. It depends on all sorts of different things. If you're in New York, for example, power rates will make the ROI for each saved watt of power return 3x more than South Carolina. Automating a manual process may save you money, unless some employees are in a union with a 3 year, no-layoff provision.

The size of your business and how you do accounting matters too. If you buy & depreciate servers, buying fewer servers + VMWare licenses will result in tax savings and reduced capital costs. But if you lease servers, those savings may be less. If you're a government agency that receives Federal funding, VMWare will save you a fortune because you won't need to run multiple servers dedicated to particular Federal programs or grants. (The Feds don't like it when you co-mingle services on hardware they fund, and expect money back)

1
vote

EDIT: I just realized how stupid those calculation look :) - I'll just leave them in, I don't think you can't do it on your own just forgot while drafting the post that I put some minor brainfarts in there too :)

Nobody here will/can give you a full description how to calculate your savings. You'd need to tell us your plan what measurements you find acceptable in "going green" (I hate that phrase btw).

Usually you should go and get the specs of your servers, for calculation purpose you to use the maximum power usage, so the energy part is dead simple.

  sum_of_kw_physical
-  sum_of_kw_virtual
-------------------
    sum_of_kw_saved
x        eur_per_kw
-------------------
          eur_saved
===================

Next up there's simple hardware cost. That's probably cost only because you usually need to get some quite big iron to run a few instances. You probably also want some central storage if you don't yet have one. If you really want to save from your old servers you could use some return program of your vendor or sell them on ebay or give them to some school (which makes the calculation really complex). Let's stay simple:

             cost_of_new_iron
+         cost_of_new_storage
- savings_from_return_program
-----------------------------
                    real_cost
=============================

Next I personally like to consider service contracts for HW to be something on it's own, because depending on how your receycling plan is you may want to get 3 service contracts for one year if your combination of options makes that cheaper, like:

  • 1st year - next business day,
  • 2nd year - same business day,
  • 3rd year - hour + replacement box

On a side note: With virtualization in place you usually want faster support contracts as less hardware needs to fail for you to go down. So that is probably going to be more expensive per server than before, depending on how many servers you save it will be cheaper or lower at the end.

depends on what you need. Another reason to not have that in hardware cost is that you are prepared upfront to get yet another year or 2 of support and know how much that costs instead of replacing the hardware if that is for some reason necessary.

consider software licenses, and say 3 months of one full time admin to be able to fully work at the same speed than before. That is not 3 months for every admin because knowledge spreads quite fast usually.

Other things you need to take into account:

  • rackspace
    • probably some savings here too
    • give (more) rackspace away to your paying customers for hard euro if you have your own DC
    • have fewer racks to pay for
    • in case of a cage consider relocating to a smaller cage

That's what I'd do for the basics in terms of "measurable" financial success or failure.

The other side would be technical success. There are a couple of metrics you need upfront:

Take this into account:

  • How long from knowing the project requirements does it take to figure out the required HW config?
  • How long from making the order call does it take for the new HW to arrive?
  • How long does it take to get a new server running (OS setup)?

Do not take this into account:

  • How long does it take to set up networking (VLAN, etc)
    • You still have to deal with switches, VLANs, cabling. I'd put that out in a seperate topic

Now consider how long it would take you in the worst case, with virtualisation you are at about the same level as without virtualisation. The worst is case is where I need to get a shiny new large server for $large_amount_of_euros just to get a single instance running. Given that you probably have to take care of the same decisions than before, delivery time and so on it takes the same amount of time.

The average case will be that you probably will be up and running in no time because all you need to do is to fire up a new instance. You still need to wait for the project to tell you what they need so that you can figure out the HW requirements. Just configure your deployment tool to set up the OS the way required boot and the rest will be taken care of by the deployment and configuration framework.

1
vote

I think that going green should be implemented holistically. Enable printer auditing and accounting and see who the big spenders are. Use XPS or PDF in email instead. I’ve seen big savings simply by examining business processes and revamping them to involve less printing, fewer servers, etc. I’ve been able to consolidate or eliminate 30% of an existing environment without spending a dime. (then you can look at licensing costs etc to determine if it makes sense to re-separate out the services in a virtual environment) it out. In addition going green means allowing folks to telecommute. That may not save the business money per se but is green and usually makes the employees happy. Also going green might cost capital now but save later. In the case where people were printing a lot it was because they all had crappy monitors, get rid of the CRT (even though it works) replace it with a larger LCD and newer Laptops/desktops and over time they will have paid for themselves in power costs.

Same goes for servers. Running a bunch of 1 u crappy CPU servers might be cheap per server but cost more than consolidating those services on to 1 expensive CPU server. Eg some older 1u dells (p4 procs) have 500w power supplies in them. A brand new bottom line ML150 comes with a 650w Power supply, but comes with roughly triple the computing capacity. Another way to save power it to have systems boot off the san. There are alot of disks spinning just to boot a system that has the other 90% on the san. Boot from san saved 1 entire chassis full of disks in the datacenter.

Bottom line- going thru all these analysis saved about 40% year over year with the CapEx paid back in about 20 months

1
  • If you are large enough start billing back printing charges to other cost centers. Jul 6, 2009 at 11:20
0
votes

The VM solution is pretty straightforward to demonstrate.

Consider the power consumption of a typical 1U server, 2 CPU sockets, lots of RAM, 3 or 4 disks at most, two redundant power supplies.

Now look at its consumption while mostly idle, and then at 100% utilization. While at 100% the power consumption is higher, the base consumption of just idling is a surprisingly high percentage of that.

Now consider consolidating 4 such servers into a single VMWare/XEN/HyperV instance on a dual 4 core box, with 2 CPU's allocated to each VM. On average the idle consumption will be higher, as it will usually be busier, but the overhead of just being on is 1/4 the amount of 4 separate physical boxes.

Everything else is gravy in terms of savings after that!

2
  • But have you actually done this and measured the success?
    – Bob
    May 31, 2009 at 21:36
  • This calculation will usually work, but you have to be sure that f.e. VM overhead isn't too much and that the big server idle consumption isn't too high (higher than 4 * idle_small_server_consumption).
    – schnaader
    May 31, 2009 at 21:40
0
votes

The larger the datacenter, the more likelihood that per-system or per-rack power savings are not a measurable quantity, and also that there isn't a managing entity that cares. My efforts to use lower-power configs and virtualization haven't been funded by any green initiatives nor have they been mandated by upper management.

0
votes

Virtualization will reduce the size of your physical server footprint, so you will get some power and maintenance savings. We saw a 62% reduction in physical servers. You will obviously not necessarily get reduced administrative savings because your just moving server images, not reducing them.

But I think the power savings of simply replacing old physicals with new are highly overrated, unless you're a super dense shop. For example, IBM claims their new x3650 M2 class box saves $100/yr in electricity. Yawn. I can't get excited about spending $3K+ to save $100/month. I suppose you could say it pays for itself in 3 years, but don't underestimate the soft costs of doing a migration to new hardware. And, new hardware tends to become more power efficient all the time without our asking for it, so my strategy is to buy new gear and virtualize when it makes sense based on business value, not just to be green.

0
votes

If you are doing any kind of server farm where you can bring individual servers online and offline then you can take advantage of Wake On Lan (WoL). When load on the existing pool is high enough then you wake up more servers. When the load is low enough you start shutting servers down. Note that this can be done in addition to most other "green" ideas. No server uses less power than one that's turned off.

0
votes

I'm not a sysadmin, but an energy consultant and software developer, but there's a story from a datacenter, that I love to tell, because it shows how far you can take it if you're determined.

I attended a conference on green IT, and there was one guy from a consulting company in Bremen, Germany, who run their own datacenter inhouse for themselves and their clients. Not a huge one, but for the type of company, it was fairly large, maybe some 50 machines. Moreover, it was constantly growing.

One day, the CEO, who was giving the talk, lost his patience with the sysadmin requesting ever more powerful cooling and ventilation gear as their server room grew and grew. These devices started to draw as much on the budget as the servers themselves. Next time the admin wanted a cooling unit, he made the guy sit down and explain all he knew about why the server room has to be at 20 °C. It turned out that nobody really knew, so they started an extensive research.

First thing they realized was that the temperature on the hardware components is important, not the room temperature. So they compiled a list of all components they were using, and got detailed heat tolerance data from the manufacturers.

Next, they made sure they had constant realtime temperature readings from right on the components, at least for a large enough sample.

Now, they slowly started to allow the room temperature to go up, keeping an eye on the components' temperature. This alone made it possible to go up to 25-30 °C. When they were slowly approaching critical temperature on the components, these were still far hotter than the room, so they tried to improve air circulation and to turn up the ventilation units. The main idea was to go as far as possible on ventilation alone, because that's way cheaper than cooling.

This process went through a couple of iterations and they repeatedly refined and upgraded their ventilation system, while raising the average room temperature. When they got to 45 °C, they began to feel their safety margin was getting too small, and they couldn't improve the ventilation further. They took a step back to 40 °C and this is now the permanent operating temperature of the data center.

They draw the air for the ventilation from outside, which helps a lot in cold German winters. The cooling has only to be switched on at all on particularly hot summer days. In winter, they reuse the heat extracted from the data center to supplement the heating system. That way, the combined energy bill for power and heating for the whole company has gone down 80%.

The server room is now very hot, and very noisy, because of the strong ventilation. They had to put extra noise insulation on the walls, to not disturb the offices next door. Admins hardly ever enter the room, because it's not exactly a great working environment. For scheduled maintenance, they start to cool down the room two days before. In an emergency, the admins put on ear protection and change into swimming trunks (no kidding), before going in.

1

Not the answer you're looking for? Browse other questions tagged .