Advice for reorganization of machine room? [closed]

Question

I am a n00b sysadmin and have been asked to completely reorganize an existing machine room with some 30-40 servers. Currently, the place is a pigsty, and wires are jumbled everywhere, so there is good reason for this.

Does anyone have any advice, tips, or maybe just interesting stories, concerning the initial setup of a machine room, that might help me avoid making mistakes, or save me a bunch of time. For example, using different colored cables for servers connected to different subnets.

Some other things I am thinking about are, how best to keep track of wires and their connections (some good piece of software?), and how best to diagram the whole thing (Microsoft Visio?).

Answer 1

Cabling plant is usually the first thing I tackle.

Here are some of my tips/tricks.

Number and Label every rack. (Also have a designation for front and back.) Know the U spaces in your racks. U1==bottom -- make sure it's easy to identify that. (you may need to tell someone someday to reboot a server which may not be labeled.)

I am a firm believer in structured rack cabling and some central patch panels.

I like to find at least two racks near the center of your area that you can bring all cabling back to..

If you have a top of rack switch, you just need enough cabling for uplinks from each rack to the center location. (maybe 4xCat5E). I always throw in some extras for growth or odd things.(serial console, management lans)

ASIDE: There are no standards to my knowledge that can use Cat6. I suggest avoiding the extra costs.

Label the top of rack uplink panel with where it's located on the other end. (Rack 20 Top goes to Rack 10, U10-12) Label each cable. Atleast with a number on each end.

So then at your center racks you can bring together all your remote racks. Plan that out in a spreadsheet.

Here's what I call "The Octopus". I intersperse patch panels that I hardwire back to a core switch in these same central core racks.

This keeps the rats nest off of the switch. You PRE-WIRE from the central switches to patch panels in the same rack, but spread them out. eg. Slot 1 in the core switch could go to U30, Slot 2, to U20, Slot 3 to U10.. Then you intersperse your rack switch feed panels. eg Rack 20 could go to U29...

This means you can get away with simple short 1' jumpers to cable all your uplinks.

It's a lot of work in the setup, but I think it's worth it in the end.

You end up having what looks like a big switchboard at the center of your space, but it'll be much more manageable if you ever need to replace a card in that central switch.

I like to establish a cable color standard. Use what you want, and make sure you can source what you need. Blue=Ethernet, White=Serial, Yellow=Uplinks, etc. Make a "key" and post it to the door and above where you store the cables so everyone knows.

Get plenty of extra cables.

Get some tubs to hold supplies.

Get a label maker, some headphones or speakers, some camp chairs, a step ladder, some trashbags, a drill with good bits for your racks/servers, a small file cabinet to stash manuals, shipping details, etc. Get a 'crash cart' or a LCD monitor mounted to a pole that you can wheel around and hook up for direct machine access.

Have a lot of fun.

Answer 2

I will give only a few pointers I feel that help on my case. I don't know how much you can or intend to spend on that, so:

• Use raised floors of good quality;
• Use a hot aisle/cold aisle organization;
• Think on a good fire detection/suppression system;
• Think on the doors and access control!
• Decide (and stick) if you want top-of-rack switching, end of row switching or something else;

The Limoncelli book has some other good tips on organizing a server room, as well as other topics that can help you.

Answer 3

A couple of suggestions:

• Grid the room out: we used two-character letter codes on one dimension and two-digit numbers on the other, so that you can quickly address a rack or piece of equipment by its location: "BG42".

• Figure out a basic cable management policy, and enforce it on pain of death. Consistency in cabling saves unplanned outages ("Oh, THAT's where that cable went!")

• Personal preference, but I suggest you run your common-access long haul cables (network, fibre, etc) above the racks in cable trays, if you have space; only run stuff under the floor tiles that you plan never to move, like major power feeds. Many an angry hour have I spent hauling out reels of old SSA cable from under the floor where it had been buried, disconnected and forgotten. In some places it became a challenge to keep the floor tiles down...

Good luck!

Answer 4

This is a huge topic, and one that will likely generate a lot of opinions. But my way may not be the right way for your situation, so I can offer only general advice.

I would first start by trying to figure out what the biggest pain-points are for the folks who use your machine room. Is it spending time to trace cabling? Is it tripping over power cords? Make a list, prioritize it, and figure out the best way to fix that particular issue. Post again here on that topic to get others opinions, and then figure out what works best for you.

I'm a little bit unclear if you're setting up a new machine room or trying to "fix" an existing one in place, though, so if you could clarify that, that would be good. The former requires a lot of planning, but there are a lot of resources available on-line to help. It's a whole field in itself, dealing with HVAC, power, space planning, etc. If the latter (fixing an existing one) I would try to work incrementally to make it better, using the pattern I mentioned above.

As far as your other questions, I don't really have anything to say about cable management, and Visio seems to be popular, although I use a Mac mostly, so I use Omnigraffle.