Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We're getting ready to move to a new colo facility and I'm designing the rack layout. While we have a full rack, we only have 12U worth of hardware right now:

  • 1x 1U switch
  • 7x 1U servers
  • 1x 2U server
  • 1x 2U disk shelf

The colo facility requires us to front-mount the switch and use a 1U brush strip, so we'll be using a total of 13U of space.

Regarding growth, I'm reasonably sure we'll be adding another 4U in servers, 1-2U of network gear, and 2-4U of storage in the mid-term. Specific questions I'm hoping to get help with:

  • where should I mount the switch? the LEDs are on top...
  • should I group the servers by function with space for adding new machines?
  • as an alternative, should I group servers based on whether they are production or staging?
  • where in the rack should I start? in the middle? at the top? at the bottom? equally spaced?

Here's a silly little ASCII diagram of what I'm thinking right now. Please feel free to tear my design apart, I've really no idea what I'm doing :) Any advice is very welcome.

edit: to be clear, the colo is providing redundant power with UPS and generator, so that's why there's no power gear in the plan, except for the 0U PDU that I didn't diagram.

    42 | -- switch ----------------------
    41 | -- brush strip -----------------
    40 | ~~ reserved for second switch ~~
    39 | ~~ reserved for firewall ~~~~~~~
    38 | 
    37 | -- admin01 ---------------------
    36 | 
    35 | -- vm01 ------------------------
    34 | -- vm02 ------------------------
    33 | ~~ reserved for vm03 ~~~~~~~~~~~
    32 | ~~ reserved for vm04 ~~~~~~~~~~~
    31 | ~~ reserved for vm05 ~~~~~~~~~~~
    30 | 
    29 | -- web01 -----------------------
    28 | -- web02 -----------------------
    27 | ~~ reserved for web03 ~~~~~~~~~~
    26 | ~~ reserved for web04 ~~~~~~~~~~
    25 | 
    24 | 
    23 | 
    22 | 
    21 | 
    20 | 
    19 | 
    18 | 
    17 | 
    16 | -- db01 ------------------------
    15 | +- disks ----------------------+
    14 | +------------------------------+
    13 | ~~ reserved for more ~~~~~~~~~~~
    12 | ~~ db01 disks ~~~~~~~~~~~~~~~~~~
    11 | 
    10 | +- db02 -----------------------+
     9 | +------------------------------+
     8 | ~~ reserved for db02 ~~~~~~~~~~~
     7 | ~~ disks ~~~~~~~~~~~~~~~~~~~~~~~
     6 | ~~ reserved for more ~~~~~~~~~~~
     5 | ~~ db02 disks ~~~~~~~~~~~~~~~~~~
     4 | 
     3 | 
     2 | 
     1 | 
share|improve this question
    
Why are you leaving holes between things? –  SpacemanSpiff Mar 2 '11 at 0:59
    
I like the visual separation. It's... not really for any good technical reason. –  bleything Mar 2 '11 at 1:24
1  
Don't leave the spaces, then. If you grow significantly you're just creating more work for yourself while serving no purpose. –  icky3000 Mar 2 '11 at 7:19
add comment

6 Answers

up vote 8 down vote accepted

There are as many ways to do this as there are opnions on the subject. My opinion is to keep it simple: firewalls, routers, and switches go at the top of the rack. The heaviest equipment goes on the bottom and then work your way up from there. Leave a U or two in the middle for a pull-out KVM tray, about waist to chest high, if you plan to put one in the rack.

Trying to populate the rack by function is probably going to lead to frustration. You've got a new file server but no room where the file servers go, etc.

I must admit that I need to keep my OC tendencies in check when I'm figuring out where to put things.

share|improve this answer
2  
Putting "like" machines together without a functional reason to do so serves no purpose and creates make-work later on. In general, always ask yourself if whatever "organization" you're trying to impose has a functional purpose. If it doesn't consider not doing it. –  Evan Anderson Mar 2 '11 at 4:15
    
+1 I do the exact same, but put the switches in back (usually in the middle somewhere). –  Chris S Mar 2 '11 at 18:27
1  
Thanks! I've realized that you (and others) are right about organization. Not worth it. New plan is to just build from the bottom up. –  bleything Mar 3 '11 at 4:27
    
Glad to help... –  joeqwerty Mar 3 '11 at 12:16
add comment

I echo the joeqwerty - Keep it simple, and label things. I do like to keep disks near the cooling and work away, with the switches on top - or using 0 U mountings on the side. You'll want to keep a U below the switch free for the cables running back to the servers (seems silly to force you to front mount a switch, but meh.)

Trying to break things out by function will just lead to insanity. The most that I would do is disks down low, 2U servers on the disks, perhaps some free space for more 2U boxes, and then 1U servers on top.

If you do a good job with the cabling, moving servers in the rack (while not a great plan) is doable.

Good luck!

share|improve this answer
add comment

If your cooling is from below (raised floor) you want your high power draw servers close to the floor. Reverse for overhead cooling.

Make sure to install blanking panels on all unused spaces.

I like to mount the switches in the middle, it keeps the cables a bit more manageable. And push back on that front mounting. The switches mount on the rear, IMHO. Can you imagine tracing a cable from front to back?

I also like to leave the top U for the connecting patch panel (drops.)

Is this a rack they are providing? Do they provide the power strips (PDUs) as well? Do the PDUs require rack units too or are they vertical mount? I'm assuming that you have two PDUs. You should verify that they are powered from two different upstream UPSes. If they are not, that might change your decision on how you are setting up the power.

share|improve this answer
    
The switch mounting policy is to ensure that hot air blows out the back, I'm not sure they're going to budge on it. I forgot to mention in my main post (but have since added) that we have a 0U PDU. Redundancy is as you describe. –  bleything Mar 2 '11 at 1:14
    
I know of more than one switch that supports reversible fan direction. –  toppledwagon Mar 2 '11 at 5:08
add comment

where should I mount the switch? the LEDs are on top...

We usually mount the switch at the top, because then any excess cable can be coiled up above the racks, hopefully in some sort of horizontal cable management system on top of the racks. It's okay to put cables on top of the racks, as long as it's done neatly and you are careful to not create a rats nest.

Putting the switch at the top means that the LEDs are hard to see unless you stand back and take a look. That's not always practical if you have narrow rows.

The colo facility requires us to front-mount the switch

So the network ports will be in front, and you'll need to loop all 24/48 cables over the top of the rack, and down the back? Sounds like another reason to put the switch at top.

Have you considered a KVM switch and a slide-out keyboard/monitor? Those should go in the middle, at an ergonomically comfortable height.

where in the rack should I start? in the middle? at the top? at the bottom? equally spaced?

Heavy machines should go at the bottom. This will save your back, and will improve maneuverability if you need to fit the big servers into the finicky rails. You never want to be put in a position where you need to hoist a heavy server above your head. Even if you have a server lift, it can be difficult to mount a server onto it's rails, and a low rack location will help you do that.

share|improve this answer
1  
Yep, ports forward on the switch. Cables come up the back of the rack, through the brush strip, and into the switch. I'll leave space for a KVM, but we don't have one now. –  bleything Mar 2 '11 at 1:35
add comment

We did this same exact thing two weekends ago, although with half the hardware.

We put all the servers and SAN at the very bottom of the rack, so it was easier to rack them because of their weight and size.

We also put the switches on the back of the rack, so there aren't cables running to the front of the rack, and everything is nice and tidy in the back.

We also had to find a spot for a KVM, monitor and keyboard, which we put right above the servers.

share|improve this answer
    
I much prefer front mount switches with a 1u pass through. It keeps things clean, and you can read cable labels and managed the switch from the front. –  Doug Luxem Mar 2 '11 at 18:24
add comment

it looks great where is the battery backup? so many servers and equipment and no seriuos battery backup?

share|improve this answer
6  
Any good colo will take care of your power for you. Typically they'll have N+1 redundant battery backups plus diesel generators. He's much better off than having a single-point-of-failure-ups. –  Jason Berg Mar 2 '11 at 0:49
    
Precisely what Jason said... the colo provides very robust redundant power with UPS and generators. –  bleything Mar 2 '11 at 0:53
    
Colo's also hate it when you put UPSes in your own rack. It really buggers up their failover capability. –  Tom O'Connor Mar 2 '11 at 1:11
2  
Using your own UPS is usually a matter of fire code. If you do allow you to add one, it needs to be connected to the EPO. –  toppledwagon Mar 2 '11 at 5:10
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.