Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I've always assumed that using Y cables from the PDU to redundant PSUs in the same server was totally acceptable practice, however I was recently speaking to someone that said that their colo wouldn't allow it.

I'm no electrician, but it would seem to me that since the PSUs are redundant, they aren't both drawing at the same time. If this is the case, it would seem that using a Y cable on two PSUs doesn't risk overdrawing the rated ampacity of the cable or individual outlet.

Have I missed something here? Why would a colo choose to ban Y-power cables in server racks?

share|improve this question
1  
Can you link to an example cable? –  ewwhite Jan 7 '13 at 15:22
9  
Hm, seems like we really need a start-to-finish "Power for Sysadmins" primer here. Some of these answers have opened a whole new can of worms –  MDMarra Jan 7 '13 at 15:53
3  
I'd strongly suspect they had some idiot in the past take a single C13 port from the PDU and "distribute" it to 5 servers or something that blew out the PDU... And now they have a rule against Wye splitters. –  Chris S Jan 7 '13 at 16:01
1  
This Q should be added to the "Canonical" list. YMMV... –  Deer Hunter Jan 7 '13 at 16:15
4  
@DeerHunter I don't think this question should be canoncial in and of itself, but a "Power for Sysadmins" primer sure seems like a good idea for a new canonical. –  HopelessN00b Jan 7 '13 at 16:36

7 Answers 7

up vote 19 down vote accepted

I am no electrician either, but I think you will at least lose the possibility of keeping your server up and running when doing so. On the contrary if you connect each PSU to a different power source, your server will still have an availble power source (hopefully).

share|improve this answer
12  
This. Y-cables don't pose an electrical hazard, they pose a redundancy hazard since. –  Lee Harrison Jan 7 '13 at 15:27
    
So you're talking about having multiple PDUs in a rack? That's not really feasible in a lot of cases, especially if you're using vertical PDUs. –  MDMarra Jan 7 '13 at 15:27
    
Depends on the rack I suppose. We are a smallish company with 2 tall server cabinets and both accomodate redundant power supplies by having independant power rails on each side of the cabinet. Of course your server room will need to be wired properly in order to take advantage of this as well. –  Lee Harrison Jan 7 '13 at 15:30
    
@MDMarra: Maybe, you will have a risk if the PSUs are working in load balancing mode and a failure occurs. In this case, the power will increase on the other still active PSU. Look at this post. –  Khaled Jan 7 '13 at 15:33
    
@Khaled but that still won't result in unsafe power consumption as far as I am aware. –  MDMarra Jan 7 '13 at 15:37

I've never used them because they're a single point of failure, at best.

Every server I deploy into a real datacenter has each PSU plugged into a different PDU in the rack, each of which are attached to a different independent UPS, on different circuits, ideally even fed from different power feeds.

If the UPS, PDU or circuit your Y-cable's attached to goes down, the redundant PSUs are going to be useless, so it seems like a waste at best, and a false sense of redundancy at worst.

EDIT:

I'll just mention that I'm talking about lower capacity 1U or 2U UPSes mounted inside the server rack, rather than the much larger, much more expensive UPS units that take a up rack all to themselves. Those are definitely built to be highly redundant in and of themselves, without the need for a secondary unit.

share|improve this answer
    
Most of the environments that I have worked in have a Symmetra or similar UPS that consumes multiple racks. I don't really see too many places having redundant UPSes of this scale, but I also don't have a lot of exposure to the "power" side of things - thus this question. I also really have never seen a failed PDU (not that it doesn't happen, I'm sure). When you hit that scale, do people really have multiple Symmetras to feed multiple PDUs? –  MDMarra Jan 7 '13 at 16:06
    
@MDMarra Yeah, see my edit, I got tunnel vision - I've yet to see anyone set up a large UPS system like the type you mention with a second unit for redundancy. If/when additional units of that size are added, it's about extra capacity, not extra redundancy. –  HopelessN00b Jan 7 '13 at 16:12
4  
@MDMarra I do a lot of datacenter design and building. It rarely is multiple UPS systems for redundancy. The big UPS systems themselves can operate with various levels of failures (batteries, power-controllers) and will fall back gracefully to just rerouting mains power if needed. Mains power will be taken over by generator if needed. So normally the UPS is just a rectifier/surge-protector and covers the take-over from mains to generator. The UPS will have multiple output circuits which feed the PDU's in the rack(s). Each server connects to 2 different curcuits. –  Tonny Jan 7 '13 at 16:37
    
@Tonny Hmmm, very good info. I wish I'd phrased the question in a way that this could be an answer and not a comment so you could get some rep out of it :) –  MDMarra Jan 7 '13 at 16:41
    
@MDMarra :-) I'm doing OK withotu the points :-) But a canonical question/answer on best practices for serverroom power seems to be in order. (And let's not forget earthing issues on that one.) –  Tonny Jan 7 '13 at 18:59

The primary reason such items are frowned upon is due to redundancy, lack-thereof. Using such a cable means all of your server's power-inputs are being fed by the same circuit so when that circuit dies (or the PDU it's connected to, I've had that happen) so does the server. Colos strongly recommend Primary and Secondary circuits for just this reason and want to see multi-PSU servers plugged into two circuits.

Way back in the day I had a group of machines that shipped with a single 3-way Y cable and 3 normal power cables for a large (7U if I remember right) 3 PSU system. The data-center I was working in at the time (this was about 1999) didn't have enough power-outlets for that kind of thing, so we ended up using the Y cable; 2 legs of the Y on one UPS, and a straight up power-cable for the 3rd PSU to the second UPS. 3-PSU systems are thankfully much less common now.

PSU Load-balancing, or is it switching?

There are differences to how power-supplies handle loading. As various power-supply benchmarks have shown, peak efficiencies are reached once you get over 50% loading. There are gains to be had for running things all on one PSU, it's more likely to be efficient. It is for this reason that some server manufacturers draw all of a server's current through a single PSU and switch to the other one when a failure happens or a whim strikes; a 230 Watt system will get best efficiency from its dual 400 Watt PSUs by running all the load through only one PSU.

Such switching systems only draw from one PSU, and therefore one circuit if using fully separated power circuits, at a time.

The downside to switching systems is that load can move unpredictably in the community of PSUs connected to a certain circuit. If enough of them throw their weight to a single circuit it can overload it. This is bad power-design, since you want to design things so that you can lose a full circuit and have things stay up, but it's still something that trips up systems engineers.

Load-balancing servers draw equal amounts of current from both PSUs. This gives predictability in circuit loading, though can still cause blown breakers if the Systems Engineers load their circuits over 50% and a circuit dies forcing the PSUs to draw 100% from one circuit, which now exceeds its rating. Again, bad power design but it's a common mistake.

Startup loads

There are two kinds of startup-loading:

  1. Everything runs flat out until BIOS (or OS Boot, or app-load) catches up and things calm down.
  2. Inrush current loading right as things get turned on.

The first is something we're all familiar with. That 120 disk SAS array may draw only 4000 Watts when running normally, but if all the disk-shelves restart at the same time it may draw 6500 Watts.

The same holds true for servers. Fans run at full speed, yes. CPUs run at full speed for a bit, yes. RAM is run at full voltage during post, yes. It's likely to draw as much as it can draw during those first stages of POST but rapidly drops off as the BIOS hands things off to the OS and power regimes take over. A server that normally draws 110 Watts during normal usage may temporarily draw as much as 200W for those first few phases.

It's this temp loading that most people think of when they say things like, "it runs the power-supplies at full on startup". Those 400 Watt power-supplies plugged into a server that draws 230W on a busy day aren't going to draw 400 Watts, they'll draw 230W... combined.

The second isn't well known, but when people run across it they get worried. This is inrush current, and takes a few milliseconds during which draw can be quite a lot higher than it normally is. The inrush current for IT devices with AC to DC converters in them (which is all of them) almost always happens twice:

  • One time when the cable is plugged in, as the pre-power stage gets power. It's this stage that allows the power-button on the front to power on the device.
  • A second time when the main distribution stage powers up and starts the device.

Because of the timings, this only becomes a factor when restoring power to a dead circuit. All those devices powering on at exactly the same time can do weird things to the power on that circuit, and that can cause damage all by itself. Doing a staged startup alleviates this.

This is the other area people think of when they say things like "power-supplies run full-tilt on startup", since each PSU has its own inrush current. But as I said, this lasts for a few milliseconds and comes in two stages.

share|improve this answer
    
Are server PSUs optimized for a different point on the load curve than consumer ones? For the latter peak efficiency is normally at ~50% and falls off on either side of that point. Ex test #3 here is the 50% point. jonnyguru.com/… Similar results were shown for both of the other reviews at the top of the main page; and on most if not all of the reviews I've seen of quality PSUs. –  Dan Neely Jan 8 '13 at 19:46
    
@DanNeely You're correct, I was remembering the curves wrong. Will update. –  sysadmin1138 Jan 8 '13 at 20:15

Our local Colo did not like them either. We had a shared cabinet and were only allowed to utilize a single PDU port for our dual PSU server. The colo didn't like it for redundancy reasons, but for a non-critical machine it was perfectly acceptable to us. There aren't any major power issues from an electrical perspective.

This is why our colo didn't like it:

  1. If the PDU dies the server is dead
  2. If the cable gets fried your server is dead

Here is why I like it:

  1. Non-mission critical box
  2. Supplied power to both PSUs giving me redundancy on the box
  3. I wasn't all that concerned with potential down-time based on the failure of the PDU, nor was I all that concerned about the cable going batshit crazy. (see #1).

The colo didn't have any issues with them electrically speaking. The draw was identical, the box only used what it needed, regardless of the number of PSUs consuming power. However, shortly after buying this cable with my Dell box Dell did stop publicly offering the cables for sale.

share|improve this answer
4  
If the PSU dies the server is dead - Did you mean PDU there? –  MDMarra Jan 7 '13 at 15:41
    
IMO, the colo/datacenter shouldn't mandate what is in essence an operational issue. If I choose to use a Y-cable then that's my choice, regardless of the redundancy or lack thereof. –  joeqwerty Jan 7 '13 at 15:52
    
@MDMarra Yes, thanks, edit made. –  Brent Pabst Jan 7 '13 at 16:12
    
@joeqwerty Fully agree, if the data center is held responsible for uptime to the box however, I would fully accept their need to mandate the use of 2 straight cables instead. That wasn't the case for me, and in a shared cabinet scenario the Y-cable was a much better choice. –  Brent Pabst Jan 7 '13 at 16:13
    
I think the "PSU" in point 3 should also be "PDU", resulting in the points meaning: you consider the risk of cable or PDU failure to be much smaller than the risk of PSU failure which sounds about right to me. I've seen many more PSU failures than cable failures. I've not done much rack-based work, so I can't really comment on PDU failure rates. –  Kevin Cathcart Jan 7 '13 at 22:21

When a server boots up, both PSU start up at full power to do a system check. This draws double the normal operating power. Now that power is more of an issue, the circuits coming into your cabinet are very specific. They don't want you to trip a breaker should you have to reboot your server.

Also, some office building don't let you do that because of fire code. You are doubling up the power that could be consumed from one outlet.

share|improve this answer
    
Do you have any documentation that shows that powering on a server with two 750W PSUs will cause 1500W of draw? –  MDMarra Jan 7 '13 at 15:52
1  
Your effect is right (or close to it) but I think your cause is wrong: Servers draw more power at startup because they have "startup stuff" to do: Spinning up disks, power on fan check, charging up capacitors, etc. - I don't know any PSU that does alias test. Doing so would require a load to test with (big honkin resistor) & switching hardware which I've never seen in a PSU. If you have seen such a beast I'd like to hear about it. –  voretaq7 Jan 7 '13 at 16:05
1  
@voretaq7 But even so, this additional load should fall well below the rating of a single PSU in the system. Sure there's increased consumption, but it's not an unsafe increase. –  MDMarra Jan 7 '13 at 16:14
    
@MDMarra That depends on how many disks you have to spin and how intelligent the controller is -- I saw a box that produced a 2-3 Amp spike (on 208V power - so ~450-500W plus the normal 150ish it ran at) because its disk spin instructions were "all 16 disks RIGHT FREAKIN NOW. (That configuration didn't last long.) Generally though you're right - we're not talking "Full rated draw on both power supplies" (at least I've never seen it be THAT bad) –  voretaq7 Jan 7 '13 at 16:43
    
@MDMarra I know from personal experience that most HP Proliant servers will briefly (3 to 8 seconds) draw full load on both PSU's on startup. Some models Dell Poweredge do it too. –  Tonny Jan 7 '13 at 16:44

A colo which is offering two redundant power circuits into your cabinet may be interested in having the two circuits balanced.

This may be why they don't like you connecting servers with dual PSUs to a single circuit, regardless if this is done with a Y-cable or with two cables plugged into the same PDU.

share|improve this answer
3  
Maybe, but then again if I'm paying for the circuits then it should be my choice how I choose to use them, load balanced or not. As long as I don't exceed the allowed capacity of either circuit the colo/datacenter shouldn't have anything to say about how I use them. –  joeqwerty Jan 7 '13 at 16:09

Being in the colo and hosting business for quite a while, I can give you the three very best reasons why a hosting company would not allow it.

Money, money and money

Think about how much a single 15 or 20 Amp circuit cost you, the hosting companies would much rather you purchase a second one for failover and charge you the big bucks twice a month instead.

share|improve this answer
    
Well, I'm not sure that covers it, honestly - you can plug two cables into the same circuit just as well as you can plug in one Y-cable. –  HopelessN00b Jan 7 '13 at 16:13
    
Sure you can, but you will eventually run out of space for your plugs. That means a second PDU. Believe me, that business is all about two things 1) Space 2) Power. Whatever will make you consume more of one or the other they will try it. –  Alex Jan 7 '13 at 16:17
    
Alright, but running your PDU out of plugs doesn't necessitate the lease of a second circuit, so I'm not sure how this is related to profit-taking by the colo/host. –  HopelessN00b Jan 7 '13 at 16:23
    
But it does make you use 1 more U if not 2. For customers renting a 1/4 or even 1/2 cabinet, that could be enough to have them upgraded to 1/2 or a full cabinet. –  Alex Jan 7 '13 at 16:26

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.