Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I would like to open a discussion that would accumulate your Linux command line (CLI) best practices and tips.

I've searched for such a discussion to share the below comment but haven't found one, hence this post.

I hope we all could learn from this.

You are welcome to share your Bash tips, grep, sed, AWK, /proc and all other related Linux/Unix system administration, shell programming best practices for the benefit of us all.

share

migrated from stackoverflow.com Sep 26 '09 at 7:14

This question came from our site for professional and enthusiast programmers.

locked by Chris S Jan 27 '12 at 4:17

This question exists because it has historical significance, but it is not considered a good, on-topic question for this site, so please do not use it as evidence that you can ask similar questions here. This question and its answers are frozen and cannot be changed. More info: help center.

85 Answers 85

It is sometimes useful to leave a program running even after you have logged out. I've seen some solutions that use nohup or even screen for that purpose. The simplest I know of is:

$ your_command_here & disown

You can also detach a running program:

$ your_command_here
# Press <Control-Z> to stop the program and bring it to background:
$ bg
$ disown
share
6  
Using screen is great because you can "reconnect" to that app from anywhere later. –  prestomation Mar 2 '09 at 21:54

Seeing color directory structures is easier.

alias ls="ls --color=tty"

Edit

alias ls="ls --color=auto"
share
5  
I prefer --color=auto because it doesn't sent rubbish control chars if redirect to file you can also do alias grep="grep --color=auto" to have matching patterns highlighted. –  kyku Mar 2 '09 at 19:51

Requirement

You need to test maximum bandwidth between 2 servers.


Solution

On SERVER1 do :

nc -u -l 54321 > /dev/null

On SERVER2 do :

dd if=/dev/zero bs=1MB | nc -u SERVER1 54321 &
pid=$(pidof dd)
while (( 1 )); do kill -USR1 $pid; sleep 2; done

You will see output such as :

182682000000 bytes (183 GB) copied, 1555.74 seconds, 117 MB/s
182920+0 records in
182919+0 records out

117 MB/s is the interesting factor here, which shows the actual network transfer bandwidth.

Explanation:

As packet will start flowing over the network you will be able to see bandwidth statistics on SERVER2, which is a pretty good estimate of the actual maximum possible bandwidth between the 2 servers.

Copying by UDP (to prevent TCP overhead).

Copying to from memory (/dev/zero) on SERVER1 to memory (/dev/null) on SERVER2, thus preventing disk I/O from becoming the bottleneck.

share
4  
Be sure that ssh is not configured to use compression in .ssh/config, 'though. Otherwise you'll get really nice numbers that are very far from the truth ;-) –  Joachim Sauer Mar 2 '09 at 20:46

When I want to make sure that I use actual command and not an alias, I use a leading backslash:

\rm -rf ~/tmp
share
4  
Well, yes unless the "command" is not alias in which case you would have to use \command rm -rf ~/tmp in that case \rm -rf ~/tmp is better –  presario Mar 3 '09 at 8:55

Use screen, a free terminal multiplexer developed by the GNU Project that will allow you to have several terminals in one.

You can start a session and your terminals will be saved even when you connection is lost, so you can resume later or from home.

share
2  
The most common switches I use are 'screen -D -R' for detaching and resuming my last screen session and 'screen -x' to view the same screen session from multiple logins. Also, get yourself a nice .screenrc from dotfiles.org/.screenrc –  Nick Devereaux Mar 5 '09 at 1:41
4  
Another nice thing about screen is that you can share a terminal with other people when you want to collaborate on something, using screen -x –  gareth_bowles Mar 12 '09 at 18:40

The command line is a funny thing. I think that you can only learn so much on your own and the rest you learn by accident watching somebody else using a command line.

I was using the shell for years painstakingly typing in directory names by hand. One day I was watching a friend mess around on a system and he kept hitting the tab key. I asked "why are you hitting tab?". Answer: it tries to complete the directory or filename. Who would have guessed--tab completion! Type a bit of the file or directory, hit tab, and it will try to finish what you typed (behavior depends on which shell though).

One day, said friend was watching me on the command line and watched me type something like:

coryking@cory ~/trunk/mozi $ pushd /etc
/etc ~/trunk/mozi
coryking@cory /etc $ popd
~/trunk/mozi
coryking@cory ~/trunk/mozi $

Who would have guessed!? He never knew about popd / pushd. Guess we are even...

share
1  
To me tab completion seems like something "obvious", very basic, but I never knew about popd/pushd either. Funny indeed. :) –  Jonik Mar 2 '09 at 20:46
3  
Also see this answer to be able to tab-complete a lot more than commands, files and directories: stackoverflow.com/questions/603696//603919#603919 –  Jonik Mar 2 '09 at 21:20
1  
That's so true. I learned about bash's backward-history-search (ctrl-r) by watching someone. I later realized that it was a readline feature and worked the same in other programs that incorporate readline (mysql, gdb, python -i, clisp etc). –  sigjuice Mar 23 '09 at 5:41
2  
How could you use "the shell" for years and not know about tab completion? –  prestomation Dec 18 '09 at 21:36

Old school, moving a directory tree from one place to another, preserving symlinks, permissions and all that good stuff:

tar cBf - . | (cd /destination; tar xvBpf -)

or across the network

tar cBf - . | rsh foo.com "cd /destination; tar xvBpf -)

New school:

rsync -urltv . /destination

or across the network

rsync -urltv -e ssh . foo.com:/destination
share
1  
-a is the same as '-rlptgoD' - that is recurse, copy symlinks as symlinks, preserve perms, preserve timestamps, preserve groups, preserve owners, copy device files and special files as device and special files. Also, between machines, -z (compress files before sending, decompress after receiving) is useful. rsync -avz [-e ssh] is your friend :-) –  dr-jan Apr 20 '09 at 21:10

This tip will make your CLI more comfortable (at least it makes for me):

create ~/.inputrc file with following contents:

"\e[A": history-search-backward
"\e[B": history-search-forward

Reload bash (eg by typing "exec bash"). When you type a prefix of a command and press the up arrow you will browse commands starting with your prefix, for example if you typed ssh it will show your former connections with remote shells. If your prompt is empty then the up arrow will browse the history the normal way.

share
3  
Usually I bind those to \e[5~ and \e[6~ (aka pageup and pagedown) instead of rebinding the arrows, but it is more convenient than Ctrl-R/Ctrl-S :) –  ephemient Mar 2 '09 at 21:06
4  
+1 for exec bash –  Dennis Williamson Sep 26 '09 at 15:01

My Favorite And Frequently Used:

List contents of tar.gz file

tar -tzf filename.tar.gz

will match line containing S1 OR S2 OR S3 OR S4

grep 'S1.S2.S3.*S4' file

Lists all subdirectories of current directory

ls -d */

Total size of directory
du -sh

find a date:

TIMESTAMP=date '+%y%m%d%H%M'

move process from foreground to background

Ctrl-z and then bg

Entire word to uppercase

echo "word" | awk '{print toupper($0)}'

Checks equality between numbers

x -eq y Check is x is equal to y

x -ne y Check if x is not equal to y

x -gt y Check if x is greater than y

x -lt y Check if x is less than y

Checks equality between strings

x = y Check if x is the same as y

x != y Check if x is not the same as y

-n x Evaluates to true if x is not null

-z x Evaluates to true if x is null

Command Line Parameters for ' test '

-d check if the file is a directory

-e check if the file exists

-f check if the file is a regular file

-g check if the file has SGID permissions

-r check if the file is readable

-s check if the file's size is not 0

-u check if the file has SUID permissions

-w check if the file is writetable

-x check if the file is executable

print first field of the last line"

awk '{ field = $1 }; END{ print field }'

important build in variables

$# Number of command line arguments. Useful to test no. of command line args in shell script.

$* All arguments to shell

$@ Same as above

$- Option supplied to shell

$$ PID of shell

$! PID of last started background process (started with &)

share

Some small pointers regarding log viewing:

  1. You can use tail -F to keep watching the log after it is truncated (by example log4j).
  2. You can use less: Open less, click SHIFT+F to emulate tail behaviour. Also useful combination is -S to disable line wrapping. less will enable you to search in the logs.
share
1  
And with tail -f you can watch multiple logs at once! E.g.: tail -f log1.txt -f log2.txt –  Stu Thompson Feb 16 '12 at 12:12

I'm moving towards never typing "rm" at a command prompt. Instead, I type "ls", and if I like the list of removed files I edit the command (easily possible with bash and ksh).

Edit to add something from the comments: "rm -i" will prompt for each deletion, which accomplishes the same purpose. Thanks!

share

How to use subversion in Linux without the help of fancy graphical interfaces that may not be available.

svn co <repo-url> . # Checks out project into current folder

svn update # Get latest changes from server

svn status # Shows what files have changed locally

svn add <path/filename> # Add some file to the repo

svn commit # Commit your changes to the repo

This holds back a lot of developers from using Linux, strangely enough.

share

Here is an excellent collection of tips I came across on digg today.

share

Use "tar xf" to extract compressed archives. The j for bzip2 and z for gzip are not necessary, as tar will detect file type of your archive. Neither is '-' sign before arguments. You will save a lot of time (over a millenium ;-)).

Better yet use unfoo to extract any archive with a single command without any unnecessary arguments.

share

Requirement: You have a directory containing large list of files you would like to delete. rm -r will fail!

example

find /var/spool/mqueue/ | wc -l
191545
rm -f /var/spool/mqueue/*
-bash: /bin/rm: Argument list too long

Solution:

find /var/spool/mqueue/ -xdev -exec command rm -f '{}' +

Explanation:

Edit: Fixing explanation following @ephemient comment.

find will supply arguments to rm by the maximum allowed arguments. This will allows rm to delete files in batches which is the fastest technique that I know of without using the -delete operation of find itself. It's equivalent to

find /var/spool/mqueue -xdev -print0 | xargs -0 rm -f

which you may find useful if your find does not support -exec ... +.

share
5  
Replace \; with +: then find will do what xargs does by default, which is batch up arguments as much as possible (without running over the argument list limit). Or just use -delete intead of -exec rm (GNU Findutils needed for both) –  ephemient Mar 2 '09 at 21:09
2  
Interesting, I hadn't heard of '+'. –  user18911 Mar 5 '09 at 3:01

Switch from bash to zsh, and see your productivity improve:

  • Really intelligent, scriptable tab completion. It completes not just command lines but all the options, names of man pages, package names (for apt-get / emerge etc), and what have you. And provides a helpful explanation of the options during the completion. All this without using any scrollback space after the command has been issued.
  • Tab completion of wildcards; write cat *.txt, press tab, and choose between any files that match the expression.
  • Switch directories just by typing their names.
  • Several line editor modes. Can behave like vi or emacs if you want it to.
  • Customizable key bindings to do whatever you wish.
  • Themeable prompts, including the ability to put prompt information on the right side of the screen and have it auto-hide when typing a long command

Google will tell you many more benefits.

share
1  
you should give more then a link. sounds interesting, but sell it to me man! –  Cory R. King Mar 2 '09 at 20:47

Install bash-completion package. It contains a number of predefined completion rules for shell. Enable it by typing "source /etc/bash_completion" if you distro doesn't do it for you. Then, for example whenever you complete after kpdf you will only see a list of directories and PDF files. It is as smart as to complete remote files after scp ssh://server/XXX (if you enabled passwordless logins).

share

Press Ctrl-R and start typing a command (or any part of it) - it searches the command history. Hitting Ctrl-R again will jump to the next match, enter executes the currently displayed command, and right-arrow (at least) will let you edit it first.

$ (reverse-i-search)`svn': svn status

I had used Linux for something like 7 years as my main OS before learning of this, but now that I know it, it's quite handy.

share

For shells like bash and ksh, you can make the command line respond to navigation commands like your text editor:

set -o emacs

or

set -o vi

will allow you to search your history and move about the command line in the way that you are used to doing in text files (e.g. in vi mode, pressing ESC then typing /string will search your previous commands for "string" - you can use n and N to move between matches)

share

I just finished reading Linux tips every geek should know at TuxRadar. Very nice articles for those who know a little bit but want to know more.

share

SSH!
SSH is the god command--I think it's the most valuable over-all command to learn. The options can be pretty daunting, but it seems like I'm constantly learning to use new command-line options for SSH that I never thought would be necessary. I may have used them all at this point.

The more you use it, the more you learn about it. You can use it to do some AMAZING things.

Note: ALL these things are doable remotely with no setup on your server except to have ssh server running.

Mount a file system over the internet

search the net for SSHFS

Forward commands.

The SVN+SSH protocol is Subversion from a remote client to a server with NO DEAMON running on it! The SVN command starts the server through the ssh shell and passes the info back and forth through the existing pipe. The rsync program does the same thing, runs against a server with no rsync deamon by starting one itself via SSH. It's easy to write your own bash files to do similar tricks.

Chain to get through firewalls

I use this all the time to jump through my linux server at home to my mac.

Forward ports:
Seems only moderately useful until you realize that you can bounce through your home firewall and configure your router at home from work as though you were doing so from within your home network).

Forward X requests:

This is another amazing one. With or without an X server running on your remote system, you can run an x-windows program and the window will appear on your local screen. Just use the switch -X, that's all!

Since you don't have to have an X server running on your remote server, the CPU impact on your server is minimal, you can have a TINY Linux server that serves up huge apps to your powerful game PC running Windows and cygwin/X.

Of course VI and EMACS work over SSH, but when I'm running at home, sometimes I want more. I use ssh -X to start up a copy of Eclipse! If your server is more powerful than your laptop, you've got the GUI sitting right there on your laptop, but compiles are done on your server, so don't worry about system load.

Run in batch files

(meaning run a local batch file that "does stuff" on other systems):

Two things combine to make this one cool. One is you can eliminate password prompts by using (more secure) encryption keys. The second is you can specify a command on the SSH CLI. I've used this in some interesting ways--Like when a compile fails on the remote server, I would have it SSH into my computer and play a sound file).

Remember you can redirect the output from the remote command and use it within your local batch file, so you could also be locally tailing a compile running on your server.

Built in to Mac

Both server and client are built into both mac and linux. In the case of the Mac and Ubuntu, enabling a server is as simple as finding the right checkbox.

On a PC install cygwin or cygwin/X (cygwin/X will allow you to forward your x-window output from your Linux machine to your Windows PC--it installs an X server)

Important Tips/config file

Never use port 22 on your firewall. You'll get a lot of hack attempts, it's just not worth it. Just have your firewall forward a different port to your server.

There are extensive configuration options that allow you to simplify your ssh commands significantly. Here's an example of mine at work:

Host home
    hostname billshome.hopto.org
    Port=12345
    user=bill
    LocalForward=localhost:1025 mac:22

When I type "ssh home" (nothing else), it acts as though I had typed:

ssh -p 12345 bill@billshome.hopto.org

and then forwards my local port 1025 to my system "mac" at home. The reason for this is that I have another entry in my file:

Host mac
    hostname localhost
    port=1025

so that once I've done an "ssh home" and still have the window open, I can type "ssh mac" and the computer at work here will actually try to connect to its own port 1025 which has been forwarded to "mac:22" by the other command, so it will connect to my Mac at home through the firewall.

Edit--cool script!

I dug up an old script I just love--had to come back and post it for anyone here who might be interested. The script is called "authMe"

#!/bin/bash
if [ ! -f ~/.ssh/id_dsa.pub ]
then
    echo 'id_dsa.pub does not exist, creating'
    ssh-keygen -tdsa
fi
ssh $1 'cat >>.ssh/authorized_keys' <~/.ssh/id_dsa.pub

If you have this script in your home directory and there is a host you can connect to (via ssh), then you can type "./authMe hostName".

If necessary it will create a public/private keypair for you, then it will ssh over to the other machine and copy your public key over (the ssh command will prompt you for a password...)

After this, the SSH command should not ask for your password any more when attaching to that remote system, it will use the public/private keypair.

If your remote computer is not always secure, you should consider setting a "passphrase" when prompted.

You may also want to configure the ssh server on the far end to not allow text passwords (only keys) for additional security.

share
4  
+1 very nice and compact post –  Karsten Mar 3 '09 at 12:20
2  
A small note on SSH over port 22: provided you've got a good root password and your users don't have terrible passwords, just running fail2ban (or something similar) is enough to keep out crack attempts. I found it really annoying to keep SSH on another port. –  David Wolever Sep 13 '09 at 16:04
1  
It can be irritating at times, but the volume of connects was just scary. I know that they aren't supposed to get through, but it's kind of like if you had someone walking up to your house and fiddling with your doorknob every minute--just unnerving. –  Bill K Sep 14 '09 at 15:49

Search in a gzipped file without first unzipping it:

gzcat someFile.gz | grep searchString
share
7  
How about zgrep, zless and zxpdf? ;-) –  Jonas Kölker Mar 3 '09 at 9:57
sudo !!

Rerun the previous command as root.

[The current top command on the site http://www.commandlinefu.com, a site themed along the lines of this question.]

share
2  
Disclosure - I run commandlinefu.com –  codeinthehole Mar 2 '09 at 21:52
9  
It would have saved this guy 16 characters: xkcd.com/149 –  Bill K Mar 3 '09 at 19:06

My personal, all time favorite: CLI meta and persistent aliases.

with these aliases, one simple command (val) to define new shell aliases, and have them going forward. concept can be extended to smaller "modes"/domains by additional alias files or aliases.

in ~/.alias.sh (make sure this gets sourced in your shell startup files)

# bash format example
alias sal='. ~/.alias.sh; echo ~/.alias.sh was sourced'
alias val='vi ~/.alias.sh; sal'

or ~/.alias.csh (csh format-- make sure it gets included in your shell start files)

# csh format 
alias sal 'source ~/.alias.csh; echo ~/.alias.csh was sourced'
alias val 'vi ~/.alias.csh ; sal'
share

grep is my friend. Seriously.

List .rb files that contain the text class foo:

grep -l "class foo" .rb

List .rb files that don't contain the text class foo:

grep -L "class foo" *.rb

List .rb files that don't contain foo or bar (you can use any regexp with -e, but you need to escape the operators):

grep -L -e "foo\|bar" *.rb
share

Most overlooked old-school command: find

Yesterday I repaired a gross permissions bug:

for u in nr dias simonpj; do
   sudo -u $u find . -type d -exec chmod g+s '{}' ';'
done
share
1  
Don't forget the -perm option to find to locate files that have permissions set a certain way. –  Dennis Williamson Sep 26 '09 at 16:07

A few useful ones:

rsync -av old_location new_location

will copy a directory structure and preserve permissions and links.

sudo updatedb && locate filename

to find files quickly (requires findutils)

apropos term_or_terms

searches the manpages.

share

Learn Vim.

It is (arguably) the best editor, but most certainly the best editor available on a bare Linux server.

share
1  
gvim is also best file browser on windows ;) –  Piotr Findeisen Jun 16 '09 at 19:12
alias rm='rm -i'
share
4  
That is a standard no-no. One day, you'll not have the alias set, and then you'll accidentally remove something precious. –  Jonathan Leffler Mar 3 '09 at 2:48

When writing loops on the command line in bash I often prefix risky commands with the 'echo' command.

for item in items; do echo something-risky; done

This way I get to see the 'something-risky' in full before committing to run it. Helps when your risky command includes variable expansions and globs. And 'set -x' is very useful when writing bash scripts.

'set -x' enables debugging. See http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_02_03.html for information on bash debugging.

share
3  
I do this too, very handy trick! –  user18911 Mar 5 '09 at 2:48
1  
+1 This can be a life saver! or data saver... –  David Oneill Nov 30 '10 at 2:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.