Here's my list of things I didn't see mentioned yet:
sticky stuff (like wall tack or putty) to pick up little bits in that fall into tight spots (little screws on motherboards)
non-oily lube (like teflon bike chain lube) for tight or frozen bolts on racked gear (esp. great for tight-back-of-rack situations where there's not much room to get good leverage)
strong knife, v. stiff putty knife, or very thin-tipped prybar for popping the head off stripped bolts. Drills work OK too but there's the vibrations and the metal shavings all over.
locking vice grips including small & needle-nosed. In some cases, almost as good as an extra set of hands.
velcro strapping not just permanent cable management but hold things out of the way in packed racks w/o creating tangles or rats' nests.
slim-edge rack tool for popping the rack nuts in and out. I keep several in my bag "just in case"...they are commonly included with new rack-mount hardware but not always and they rarely are handy when you have to move something months or years later. Saves me a lot of ripped up fingers. They are also often just right for depressing the lock-clip on a tight network cable (esp. one with a boot) or a lock-clipped fibre patch for those of us with stubby or snausage fingers.
another use for a cellphone: some (easy) way to light up the screen as a quick flashflight for checking stuff out in the backs of cabinets
Back in my desktop days, I had kept a handful of CMOS batteries in my bag. This was esp. helpful when dealing with labs and offices where the PCs were 2+ years old and resolved many "head scratchers" in short order.
++ paper clip...tape monkey's best friend when a robotic library is good and jammed :D