41
votes

I'm wondering about tools that are powerful and that most system administrators either don't know about or don't use (but should).

For one thing, I like the possibility of finding out about a tool that is good and that I should be using - or at least, trying out. I also find that giving these tools their time in the sun (again) can be a positive, letting others know about the wonderful tools that are out there.

Thus, things like sudo, vi, emacs, dtrace, ps, and top are out. I have some ideas but I just hate to skew the statistics...

I'll just wait and see if anyone mentions my favorites.

7
  • 7
    There are so many 'what tools' questions, do we really need another?
    – Zoredache
    Jun 6, 2009 at 9:49
  • 4
    I just have to ask: if no one's heard of a tool, then how will it get upvotes?
    – quux
    Jun 7, 2009 at 2:52
  • 5
    +1 for camera phone. They are also incredibly useful when working with someone over the phone to troubleshoot a problem and there is a long error message on the screen. I wish all remote hands people had/made use of them.
    – Mark
    Jun 17, 2009 at 18:40
  • 1
    I've used it to document lights on the front panel and ports on the back panel (in this case, of a DEC Alpha system).
    – Mei
    Jun 17, 2009 at 23:18
  • 2
    Please mention one tool per answer. Aug 4, 2009 at 14:18

74 Answers 74

4
votes

Someone bought a very expensive scissor-jack that can lift about 300Kg to a height of 3m - a waste of money I thought - until we needed to fit 5 HP C7000 blade enclosures into one rack - boy it came in useful for that top one :)

4
votes

Here's my list of things I didn't see mentioned yet:

sticky stuff (like wall tack or putty) to pick up little bits in that fall into tight spots (little screws on motherboards)

non-oily lube (like teflon bike chain lube) for tight or frozen bolts on racked gear (esp. great for tight-back-of-rack situations where there's not much room to get good leverage)

strong knife, v. stiff putty knife, or very thin-tipped prybar for popping the head off stripped bolts. Drills work OK too but there's the vibrations and the metal shavings all over.

locking vice grips including small & needle-nosed. In some cases, almost as good as an extra set of hands.

velcro strapping not just permanent cable management but hold things out of the way in packed racks w/o creating tangles or rats' nests.

slim-edge rack tool for popping the rack nuts in and out. I keep several in my bag "just in case"...they are commonly included with new rack-mount hardware but not always and they rarely are handy when you have to move something months or years later. Saves me a lot of ripped up fingers. They are also often just right for depressing the lock-clip on a tight network cable (esp. one with a boot) or a lock-clipped fibre patch for those of us with stubby or snausage fingers.

another use for a cellphone: some (easy) way to light up the screen as a quick flashflight for checking stuff out in the backs of cabinets

Back in my desktop days, I had kept a handful of CMOS batteries in my bag. This was esp. helpful when dealing with labs and offices where the PCs were 2+ years old and resolved many "head scratchers" in short order.

++ paper clip...tape monkey's best friend when a robotic library is good and jammed :D

3
votes

One tool that I absolutely find indispensible is tcptraceroute 1 - this is a traceroute that does not use ICMP packets to perform timings, but uses TCP instead. This allows you to traceroute without regard to the typical blocking of ICMP: it works well.

1
  • tcp traceroute needs also ICMP fore the enswers Sep 12, 2011 at 17:46
3
votes

I used this much more as a helpdesk support person than an SA, but I always keep a push/pull spring hook in my toolkit. It's a hook on one end, and a sort of s shape on the other. Perfect for getting bent pins on connectors straightened out.

The only link I can find to one is this: spring hook - but it should give an idea of what it looks like.

3
votes

A good knife. Not a multi-tool, a KNIFE, as in something made by SOG, Cold Steel, or Boker. You'd be astonished at how useful it is.

3
  • +1 on the knife. I prefer the spring assisted kind much easier to open when you other hand is tied up holding something in place.
    – Zypher
    Jun 17, 2009 at 19:54
  • Zypher -- I'd have suggested that, except those are illegal in some states and municipalities.
    – SilentW
    Jun 17, 2009 at 22:33
  • A great compromise is the Benchmade Osborn Axis. It uses a bearing system, opens just as fast (with one hand), and can also be closed (with one hand). Jun 17, 2009 at 22:38
3
votes

I once used office chairs in place of one of the fancy scissor-jack devices Chopper3 mentioned. I needed to relocate the UPS's in a rack, I do not know why they were half way up in a wheeled rack, but they were. I'd scheduled the maintenence window on a Sunday. My helper had a family emergency, and could not make it. I had my 9 year old duaghter with me. I could not manhandle the UPS to the ground by my self, and she wasn't strong enough. So I pulled the UPS out as far as I could on the rails, but a chair under it, and had her hold the chair while I lowered the UPS on to the chair. After resting and re-gripping, I lifted the UPS, and she pulled the chair out, so I could lower it the rest of the way down onto a set of blocks I'd brought to help hold the bottom UPS at the right height for the bottom of the rack. Then the remaining UPS's when onto the stack so they where roughly at the right heights as well.

1
  • Had the same experience ... sans 9 year old ... Jun 18, 2009 at 9:10
3
votes

Here is a few tools I have come across:

SYDI

Auto network documentation tool.

Orca

Messing with .msi files.

RichCopy

An advanced alternative to Robocopy.

ImgBurn

Best free burning software hands down.

PowerGUI

A nice PowerShell editor.

Your Brain!

A lot of people seem to forget they have one.

3
votes

Voice coil magnets taken out of dead hard drives. In case you don't know, these are very strong rare earth magnets. I use them for multiple things:

  • Wiping hard drives that are being tossed out that I don't have time to take apart. They're strong enough to wipe the servo tracks from the drive, making it essentially unusable.
  • Holding notes or wires in strange places: most of the magnets have holes in their backing plates that you can thread a cable tie through and they stick to any small bit of steel.

I've used CDs as mirrors to see behind boxes (usually desktops crammed under someone's desk) when I didn't want to pull them out.

1
  • I've used CDs/mirrors as lights for back under a desk to see behind a tower PC pretty frequently. I hate the blue burned CDs, since they don't work at all for this. Jun 26, 2009 at 17:11
3
votes

Absolutely vital: A dentist mirror for those tight spaces!

2
votes

For Windows:

PATHPING is nice (and built in) for some advanced ping stats

I also like QCHECK : http://www.ixchariot.com/products/datasheets/qcheck.html It's a great GUI based simple network analyzer.

File Unlocker is a life saver at times even on servers: http://ccollomb.free.fr/unlocker/

2
votes

fldiff is a graphical diff program

2
votes

All our apps are clustered, so cssh is pretty handy.

scp - only mentioned because so few people use it, relying on FTP instead. I've always loved the ability to grab files instantly from remote hosts. Secure, quick, and doesn't need a dedicated daemon running.

2
votes

In my server closets, I always put one of those cheapie push-on battery operated lights. It's great if(when) there's a power outage.

1
  • Even better is to keep one in your pocket or on your keys. They come in handy for much more than power outages.
    – Mark
    Jun 18, 2009 at 2:58
2
votes

Swiss Army Knife.

I have two, a big one I keep in my desk and a standard one on my keyring (just remember to take it off when you go to the airport!).

It even saved my life (or at least bad trip to the hospital) one day when I was cutting through some 240v wiring our electrician swore was disconnected (but wasn't).

5
  • OK, I'll bite, how'd the knife save your life if you where using it to cut through live 240V wires?
    – BillN
    Jun 17, 2009 at 23:56
  • 1
    Haha, well the story itself is a bit too long to post in the answer (short and concise is usually best), but here's the abridged version: Electrician swore that he had disconnected all the wiring in the room we were demolishing. We found a mains cable inside one of the walls that hadn't been removed (electrician was supposed to do this), and after confirming multiple times that yes, it was disabled, my co-worker left to get his pair of pliers, made out of aluminium. I couldn't be bothered waiting for him, so I popped out the knife, grabbed the mains line Jun 18, 2009 at 0:49
  • 1
    (it was two heavy guage cables, individually insulated) and ripped through it. There was a huge BANG, sparks everywhere, and my knife has a nice hole in the middle where the contact was made. Because the knives have a plastic shroud, I had a very small contact area (not that it would have mattered), but the main reason was that if I had waited for the aluminium pliers, which were not insulated at all, I would have just snipped right through both of the live lines simultaniously and there would have been no protection between me and the two active lines. We sued the electrician after that. Jun 18, 2009 at 0:52
  • Its worth mentioning that this was a shoddily-built building and that the circuit that line was on had been removed from the ground breaker circuits (illegal in Australia). Jun 18, 2009 at 0:54
  • On this note, I have a fantastic pair of VDE insulated pliers and wire cutters. bit.ly/9LNBij Mar 10, 2010 at 15:51
2
votes

12" prybar I have used it for lifting stuck tiles, grabbing cable bundles just out of reach, drifting a half loaded rack over a half inch so it was squarely on the correct tile.

Vice Grips Handy for extracting mounting screws some other monkey stripped out with a power drill.

The little slim jim A piece of bent spring steel that has one end about 3mm narrower than a square rack hole. This little guy with a hook in the end makes setting clip nuts in and getting them out a dream.

Rubber Chicken, and a Sledgehammer. I menaced a recalcitrant server with them during a reboot late one afternoon after a long and hard day of troubleshooting a problem the vendor had NO clue about. It lead to the best moment of my day when I scared the crap out of my director by walking into my managers office where they were meeting returning the sledge. Director asks "What's the sledgehammer for?" I said "I used it to fix the ServerX," The best part was, that menacing the server worked (that or the 5th reboot as recommended by support with nary a config change...did the trick)

1
  • 1
    +1 for proper use of Rubber Voodoo Chickens. For those who do not have such devices, they are an Appropriate Rubber Chicken Device, which is held neck in left hand, feet in right hand, and upon hearing the lament of users, is waved over the machine(s) in question, chanting, "Oh mightly rubber voodoo chicken, we beseech your rubbery quivering blessings upon this machine!" (followed by a reboot). Tends to work better than 50% of the time and the users are amazed, which has lead to an increase in the number of user-owned rubber chickens in the office. Jun 18, 2009 at 19:21
2
votes

Sledgehammer... for things that aren't working by around 4:45.

(from coworker)

2
votes

ratchet and webbing (kinda like a tie-down for a truck) to 'lift' servers high enough to place on shelves or racks.

cart/wheeled chair for a 'crash-cart' console when you don't have a Cyclades port for each server.

indoor/outdoor thermometer to check for periodic hot-spots suspected in the datacenter.

garage-style retractable power-cord located semi-centrally for when you need power but dont want to hunt under the floor for a receptacle/plug.

2
votes

Patience.

I know it sounds corny, but it takes a lot of willpower to just sit on your hands and wait or refrain from use of excessive strength (how many connectors have you destroyed due to losing patience and pulling harder?)

2
votes

A MagnoGrip (http://www.thinkgeek.com/gadgets/tools/917f/). It holds onto screws and other small metal items for you. I tend to leave one stuck to the first cabinet in the room.

1
  • This seems like a magnetic catastrophe waiting to happen. +1 for using magnets around open servers! Jun 8, 2010 at 19:42
1
vote

grep, awk, netcat - can't live without them

For monitoring I use NetMRG and Nagios.

Uptimed is also a nice tool to monitor uptime and avalibility.

1
  • You might want to take a look at Ncat (part of Nmap 5). It's an updated, more full-featured netcat. Oct 30, 2009 at 20:34
1
vote

I try not to use too many "nonstandard" tools. That is, tools that aren't easily available between operating systems or out of a particular distribution's software repository. I work on a Macbook, and I do testing on a variety of different Linux/Unix platforms.

That said, I like ohai(1). It is the node data collection tool used by Chef. It outputs data in JSON, so it can be manipulated with a variety of JSON parsing libraries. Since I work on Chef :-), it's a "nonstandard" tool available on every system I work with.

(1) Disclosure. I work for the company that wrote Chef and Ohai.

1
vote

Odd answer in a way as it's not a tech tool directly - but I couldn't live without EverNote. It's a centralised note taking system, it has Windows, Mac, iPhone clients and lets you clip pits of web pages (with links to the original), has catergories, records images & voice notes.

It's what I use when my brain breaks :)

1
  • Nicest thing about EverNote is that it OCRs everything you capture. This makes it easy to find stuff later, or to copy content from images, etc. Jun 12, 2009 at 21:07
1
vote

It's not exactly a small cli tool as such, but I can heartily recommend Zenoss Core for network and system monitoring. It has restored my faith in monitoring-and-alerting software.

Where tools like Nagios, Cacti, etc. all have steep learning curves and seem to be good at only one thing, most often you have a need for an all-in-one solution, and Zenoss Core provides that for free (and Free).

It's hard to summarize, but basically it's an application with web-interface that autodiscovers devices on your network, then monitors tons of parameters, logfiles, ports, services, software, hardware, amount-of-pages-printed, etc. If anything fails or crosses a threshold you get an alert. It's easy to get started: enable/install snmp on each device you want to monitor, install Zenoss on a server, and open the web interface.

The free Core version is good enough for small and medium businesses, while the commercial Professional and Enterprise versions provide some advanced features.

If you ever thought about Nagios, Cacti or the like, be sure to evaluate Zenoss as well.

1
  • +1 for this package, used it for a short while and it's got a bright future. Aug 4, 2009 at 12:01
1
vote

I personally never go anywhere without UnxUtils. It's got a very large amount of Nix command line utilities. Some of my faves include: find, wget, and sed. And of course it's nice to not have to remember to type dir/copy/etc when I'm switching to a Windows machine after a long stint on a nix machine.

Throw in a batch file to quickly add it to your PATH and you're good to go.

http://sourceforge.net/projects/unxutils/

1
vote

sdiff. It displays two files in a split screen, showing the direction of change in the middle. It can also merge files interactively.

#sdiff -w 40 /etc/fstab /tmp/fstab
/dev/md0                /dev/md0
/dev/md1                /dev/md1
tmpfs                   tmpfs
devpts                  devpts
sysfs                   sysfs
proc                    proc
LABEL=SWAP-sdb2    |    /dev/sda2
LABEL=SWAP-sda2    <
1
1
vote

I've once used a butane soldering iron to reseat a flapping component from a PCB whilst in the field.

It worked, I might add. But I avoid repeating if possible.

1
vote

Paperclip (other use) - if a system is hung (particularly on a single processor, single core, CPU) and you don't want to reboot it, and all else fails, shove a paper clip in a USB port. I've found this to create a fault that allows the CPU to handle user input.

Rubber band - our office has dozens of printers, many of which are connected locally to their computers for security reasons. The parallel cables often pop out ... tying a rubber band around them holds them in place.

Fist - a quick slap on a rattling AC unit, or even a spotty fax, is often all that is needed to keep it going until a Service Tech can arrive.

Old analog answering machine - we have a few analog lines coming into our data center for telecom system for backup purposes ... surprising how often Verizon forgets to reroute things when the T1 is down so we just hook up the old Record-a-Call so clients don't get a dead ring.

Awl - basically an ice pick. Will punch a hole through anything. We use them to destroy old hard drives, open up a machine with stuck screws, and other uses.

1
  • Mmmm, percussive maintenance... Jun 18, 2009 at 12:37
1
vote

Did anyone mention sharpies? You can write on anything with them!

1
vote

I use this almost daily a good ol' leatherman tool for 12 years. (sorry can't post images )

It can cut cables, turn screws, cut cardboard, plastic and wood... Waaaay better than a swiss army knife IMHO.

2
  • An imperfect tool for an imperfect world. Jun 22, 2009 at 12:56
  • Excuse me but imgur has deleted the image.
    – hsym
    Mar 18, 2011 at 0:35
1
vote

dsh

Distributed SHell. With dsh you can perform a command via ssh on multiple servers. I find it quite handy when doing a lot of the same things on a serverfarm, at least when it's not complicated. For instance, doing a reboot of the whole farm is just 'dsh --all reboot'. I wouldn't recommend it for using it interactively. With dsh you can make lists of servers, like a list of all your webservers or all servers located in .uk, and only perform the action on that list.

ClusterSSH

Sort of the same, it spawns multiple ssh sessions and you can input in multiple sessions at the same time. It's extreme usefull when you've got interactive commands, like aptitude dist-upgrade. I found this very usefull when upgrading a serverfarm from etch to lenny.

Not the answer you're looking for? Browse other questions tagged .