Tools a Unix administrator cannot live without [closed]

Question

After administering Unix or Unix-like servers, what tools (command-line preferably) do you feel you cannot live without?

Answer 1

GNU screen - essential when you're managing large numbers of systems and don't want to have a dozen terminal windows open.

Answer 2

Some I know that I cannot live without...

• tee - allows simultaneous writing to STDOUT (standard output) and a file. Great for viewing information and logging it for later.

• top - the task manager of UNIX, gives a great overview of the system.

• tail -f - allows you to view appended data as a file grows, great for monitoring log files on a server.

• grep - Global Regular Expression Print, great for searching the system for data in files.

• df - reports disk usage of current filesystems.

• du - reports disk usage of a certain file/directory.

• less - needed to view man pages! also useful for viewing output of commands in an easily seekable manner.

• vim/Emacs/nano/pico/ed - whatever your text editor of choice may be, self explanatory of why it's needed.

Answer 3

lsof to determine which processes are using a file or directory (useful when trying to figure out what is preventing a device from being umount'd)

netstat to determine which processes are using network connections (especially useful when trying to figure out which daemon is bound to a certain port)

Answer 4

Learn all the basic tools, but learn Perl.

Perl is ideal for manipulating text, and since un*x operators live on text files, pipes, input and output, Perl is a great fit.

The added bonus is Perl is cross platform and if you have to do some work on a windows box you have an easily installable (just drop a Perl directory on the server) language that you already know.

And on that train of thought, get Cygwin as well. If you are a un*x admin and have to work on a windows box (even your desktop) having ls, rm, grep, sed, tail etc save you a lot of time when switching OS's.

Answer 5

• sed
• awk

The forgotten grandfathers of modern systems scripting. I know Perl gets most of the love (along with Bash scripting, Python, Ruby, and [insert your favorite scripting language here]), and don't get me wrong, I love Perl. I make use of it almost daily.

But sed and awk should not be forgotten, overlooked, or ignored. For a lot of cases, sed and awk are the best tools for the job. Quick examples are command line filtering with sed, and quick and dirty log processing with awk. Both could be done in Perl, but will require more work and development time.

Answer 6

[rsync][1], especially in concert with ssh. It allows simple efficient copying of files from host to host. How did we ever cope without ssh and rsync? :-)

Answer 7

Netcat.

• Test if TCP services are listening.
• Perform transactions against plaintext protocols such as SMTP.
• Quick insecure data transfers between machines.
• Telnet client emulation.

The network swiss army knife, as they say.

Answer 8

Face it - sooner or later you'll deal with the network as well. mtr, tcpdump and tshark are really useful for seeing what's happening.

Answer 9

For quick scripts, automation, etc:

• bash
• perl

To connect to your *NIX server:

• Open SSH (Linux client)
• Putty (Windows client)

Answer 10

I use most of the tools already listed, but here's one no one has touched on yet:

Puppet - system for automating system administration tasks

Answer 11

For scripting:

• python
• gawk

Answer 12

Most of the standard ones are included in other answers, so I'll go fo non-standard ones:

• htop — great for process management;
• pinfo — lynx like browser for info and man pages.

Answer 13

A couple of handy tools I haven't seen mentioned yet:

• dstat --nocolor (overview of cpu-, disk-, net-usage)
• iftop (nice dynamic overview of network traffic)
• ccze (colour logfiles nicely)
• ssh tunnels (can be useful once in a while; see the manual; -R)
• expect (automate interactive, chatty dialogy interfaces, nice if you're in a pinch)

Answer 14

ClusterSSH

ClusterSSH controls a number of xterm windows via a single graphical console window to allow commands to be interactively run on multiple servers over an ssh connection.

Answer 15

vmstat 1

Gives you a great overview of system behaviour.

Answer 16

pv: Displays the progress of long operations that can be redirected. http://www.ivarch.com/programs/pv.shtml

Useful then you want to monitor something that is going to take ages, like copying/compressing a raw block device over the network (which is how I take paranoia backups of my 8Gb netbook before tinkering with anything major like tweaking with file system settings).

Also: I'll second votes for ssh, rsync, screen, htop and netcat as mentioned by people above - all of which are more important than pv but pv had not been mentioned yet. In fact pv is often a useful addition when piping stuff to or from to netcat.

Answer 17

ssh, Vim, htop, su, Python, ls, cd, screen, du, tar :)

Answer 18

sudo.

Seriously though, tail -f is useful.

Answer 19

Some that haven't been mentioned before:

• head/tail
• diff
• pstree
• tar
• gzip/bzip
• watch

Answer 20

zsh as a shell

It is especially efficient with grml.org's extensions/setup.

Answer 21

Most of these tools are made much more powerful using Bash "programmable completion" - so you can tab-complete things like commandline options, or say the name of a package with "apt-get install". It will also limit what you tab-complete for relevant files - for example, "unzip" will only complete supported archive files.

It really is the mutts - if you have never tried it you probably just need to fiddle your .bashrc:

if [ -f /etc/bash_completion ]; then
    . /etc/bash_completion
 fi

Certainly this is true on Ubuntu and Debian. You may need to get the package on some Linux distributions.

Answer 22

Some additional answers can be found in this similar question

Answer 23

tar pipe!

piping the output of tar to another utility, to tar running on the same box, or to tar running over SSH is my favorite old-school Unix move for moving files from one place to another.

This also gives you the Windows-style option of copying one folder to another and ending up with all of the files in the source and destination directory.

Answer 24

man - to read the man pages.

elinks - to check google, cause I sure as hell cant remember everything.

And attention to detail & tenacity, because without them I just waste time.

Answer 25

iotop, is a top-like program to monitor I/O accesses to your disks.

Answer 26

vi

find

ssh

AWK

sed

netcat

tar

ps

Answer 27

Simple, basic but still essential:

ps - report a snapshot of the current processes.

free - Display amount of free and used memory in the system.

w - Show who is logged on and what they are doing.

Answer 28

pkill or ps for killing processes.

If you want to use ps to kill any process with a given name or under a certain directory blah (or any matching string you require) you can:

kill `ps -ef | grep <blah> | grep -v grep | awk '{print $2}'`

Answer 29

A few things overlooked I wanted to mention.

• vim -d split screen console diff that makes it very easy to see the differences in a file
• pdsh allows you to easily run a command over as many systems as you want either serial or parallel(I am a cluster admin. I can't function without it.)
• nmon is like top on crack. It gives you a great idea of what is going on on a system on a single screen. You can see disk I/O, network I/O CPU usage, and memory usage real time. At the very least a real fun thing to play with when profiling a system.

Oh, and I forgot to mention, when scripting, I believe you should always use Korn. I hate Korn(Not the band. I love the band:-P) but it's literally everywhere. You can take a script and move it between Solaris, AIX and Linux and not have to worry about whether or not the admin had the decency to install Bash.

Answer 30

One tool sometimes very handy is nohup. I use it to run scripts that last for a long time using remote SSH clients.