Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am a n00b sysadmin and have been asked to completely reorganize an existing machine room with some 30-40 servers. Currently, the place is a pigsty, and wires are jumbled everywhere, so there is good reason for this.

Does anyone have any advice, tips, or maybe just interesting stories, concerning the initial setup of a machine room, that might help me avoid making mistakes, or save me a bunch of time. For example, using different colored cables for servers connected to different subnets.

Some other things I am thinking about are, how best to keep track of wires and their connections (some good piece of software?), and how best to diagram the whole thing (Microsoft Visio?).

Thanks!!

share|improve this question
2  
would suspect that you could virtualize probably 80% of your servers, so thats my best advice for reorganizing this thing! –  tony roth Mar 23 '11 at 0:18
1  
post pictures! :) –  toppledwagon Mar 23 '11 at 2:33
add comment

4 Answers

up vote 3 down vote accepted

Cabling plant is usually the first thing I tackle.

Here are some of my tips/tricks.

Number and Label every rack. (Also have a designation for front and back.) Know the U spaces in your racks. U1==bottom -- make sure it's easy to identify that. (you may need to tell someone someday to reboot a server which may not be labeled.)

I am a firm believer in structured rack cabling and some central patch panels.

I like to find at least two racks near the center of your area that you can bring all cabling back to..

If you have a top of rack switch, you just need enough cabling for uplinks from each rack to the center location. (maybe 4xCat5E). I always throw in some extras for growth or odd things.(serial console, management lans)

ASIDE: There are no standards to my knowledge that can use Cat6. I suggest avoiding the extra costs.

Label the top of rack uplink panel with where it's located on the other end. (Rack 20 Top goes to Rack 10, U10-12) Label each cable. Atleast with a number on each end.

So then at your center racks you can bring together all your remote racks. Plan that out in a spreadsheet.

Here's what I call "The Octopus". I intersperse patch panels that I hardwire back to a core switch in these same central core racks.

This keeps the rats nest off of the switch. You PRE-WIRE from the central switches to patch panels in the same rack, but spread them out. eg. Slot 1 in the core switch could go to U30, Slot 2, to U20, Slot 3 to U10.. Then you intersperse your rack switch feed panels. eg Rack 20 could go to U29...

This means you can get away with simple short 1' jumpers to cable all your uplinks.

It's a lot of work in the setup, but I think it's worth it in the end.

You end up having what looks like a big switchboard at the center of your space, but it'll be much more manageable if you ever need to replace a card in that central switch.

I like to establish a cable color standard. Use what you want, and make sure you can source what you need. Blue=Ethernet, White=Serial, Yellow=Uplinks, etc. Make a "key" and post it to the door and above where you store the cables so everyone knows.

Get plenty of extra cables.

Get some tubs to hold supplies.

Get a label maker, some headphones or speakers, some camp chairs, a step ladder, some trashbags, a drill with good bits for your racks/servers, a small file cabinet to stash manuals, shipping details, etc. Get a 'crash cart' or a LCD monitor mounted to a pole that you can wheel around and hook up for direct machine access.

Have a lot of fun.

share|improve this answer
    
Thanks! This is great advice, I appreciate the tips. –  emish Mar 25 '11 at 16:26
add comment

I will give only a few pointers I feel that help on my case. I don't know how much you can or intend to spend on that, so:

  • Use raised floors of good quality;
  • Use a hot aisle/cold aisle organization;
  • Think on a good fire detection/suppression system;
  • Think on the doors and access control!
  • Decide (and stick) if you want top-of-rack switching, end of row switching or something else;

The Limoncelli book has some other good tips on organizing a server room, as well as other topics that can help you.

share|improve this answer
    
Your first 4 points are excellent suggestions if you're building a server room or data center but are probably net very applicable or practical regarding reorganizing an existing server room. –  joeqwerty Mar 23 '11 at 1:59
    
@joe he asked for initial setups :) –  coredump Mar 23 '11 at 2:06
    
I think he meant starting from scratch with what he's got. He did state it was an existing server room. Nonetheless, your suggestions definitely should be considered and implemented if practical in this situation. –  joeqwerty Mar 23 '11 at 2:29
    
Sorry for any confusion, but I mean to reorganize an existing room of servers. I mentioned 'initial setup' because everything is going to be reorganized (ie. moved, rewired, etc..) –  emish Mar 23 '11 at 5:11
add comment

A couple of suggestions:

  • Grid the room out: we used two-character letter codes on one dimension and two-digit numbers on the other, so that you can quickly address a rack or piece of equipment by its location: "BG42".

  • Figure out a basic cable management policy, and enforce it on pain of death. Consistency in cabling saves unplanned outages ("Oh, THAT's where that cable went!")

  • Personal preference, but I suggest you run your common-access long haul cables (network, fibre, etc) above the racks in cable trays, if you have space; only run stuff under the floor tiles that you plan never to move, like major power feeds. Many an angry hour have I spent hauling out reels of old SSA cable from under the floor where it had been buried, disconnected and forgotten. In some places it became a challenge to keep the floor tiles down...

Good luck!

share|improve this answer
add comment

This is a huge topic, and one that will likely generate a lot of opinions. But my way may not be the right way for your situation, so I can offer only general advice.

I would first start by trying to figure out what the biggest pain-points are for the folks who use your machine room. Is it spending time to trace cabling? Is it tripping over power cords? Make a list, prioritize it, and figure out the best way to fix that particular issue. Post again here on that topic to get others opinions, and then figure out what works best for you.

I'm a little bit unclear if you're setting up a new machine room or trying to "fix" an existing one in place, though, so if you could clarify that, that would be good. The former requires a lot of planning, but there are a lot of resources available on-line to help. It's a whole field in itself, dealing with HVAC, power, space planning, etc. If the latter (fixing an existing one) I would try to work incrementally to make it better, using the pattern I mentioned above.

As far as your other questions, I don't really have anything to say about cable management, and Visio seems to be popular, although I use a Mac mostly, so I use Omnigraffle.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.