112
votes

I would like to open a discussion that would accumulate your Linux command line (CLI) best practices and tips.

I've searched for such a discussion to share the below comment but haven't found one, hence this post.

I hope we all could learn from this.

You are welcome to share your Bash tips, grep, sed, AWK, /proc and all other related Linux/Unix system administration, shell programming best practices for the benefit of us all.

9
  • Why Linux only? Many tips may be useful on almost all flavours of Unix.
    – mouviciel
    Mar 2, 2009 at 19:49
  • This question might be a tad broad - there is enough to Linux/Unix command line best practices to fill entire books...
    – Jonik
    Mar 2, 2009 at 19:50
  • I agree that unix tips that can be applied to linux also are very much welcome, I've written linux because A. This is what we work with here. B. because it's gaining a more wider public exposure then unix in recent years. Mar 2, 2009 at 19:57
  • I think it's a great idea to comiple a list of the most useful and interesting commands that people use.
    – Shane
    Mar 2, 2009 at 19:57
  • Are any of these relevant? refspecs.freestandards.org
    – S.Lott
    Mar 2, 2009 at 19:58

85 Answers 85

7
votes

$_ is an environment variable for the last argument from the previous command

So if I create a new directory

$ mkdir ~/newdir/

To enter I simply go

$ cd $_

This is handy for complicated and large texts such as URLs, directories, long file names etc.


You can also refer to each argument in the previous command using !:{number}

$ echo !:0 !:1 !:2

Note that bash will expand this before you execute it (to see this press up to go through your history).

$ touch one two three
$ ls !:1 !:2 !:3 

Unlike $_ which is an environment variable, this will expand to 'ls one two three', perform the action and print the command to the shell. This method is a lot harder (in my opinion) to use than $_ which I use much more frequently.

Note: you can also use !$ instead of $_ but the former will expand

1
  • 2
    A nice shortcut for getting the last argument of previous command(s) is M-. (aka (left) Alt-.) This will insert the same stuff as $_ with a single key press. Pressing it repeatedly will bring last arguments from previous commands. There's also a way of inserting other arguments,but I'm short of time
    – kyku
    Mar 5, 2009 at 8:09
6
votes

There was an answer about using pushd/popd to remember directories. If you want to temporarily visit some directory, you can save typing by using cd - command, like this:

/home/eugene $ cd /etc
/etc $ cd -
/home/eugene $

Also, one of my favorite commands is xargs. I use it very often with find, but it can be also handy in other situations. For example, to find which command line arguments were used for starting some process, you can use the following command on linux:

 $ xargs -0 echo < /proc/[PID]/cmdline

In some cases (especially when working with source code) ack is more handy than grep because it automatically searches recursively and ignores backup files and version control directories (like .svn, .hg). No more typing long command lines like find . -name \*.c | xargs grep 'frobnicate'.

1
  • To temporarily visit a directory and jump back automatically, use a subshell: (cd test; dosomething) leaves you in the directory you started from. Sep 26, 2009 at 16:13
6
votes

A nice one I've seen today from a friend.

Clean the log of the application (for next launch and co.)

> /var/log/appname.log

(Note the the > is part of the command).

This is the same as doing:

echo '' > /var/log/appname.log
2
  • That's kind of scary, actually... especially when you copy/paste from a terminal or website where the prompt ends with '>'...
    – Thomas
    Mar 14, 2009 at 6:18
  • The two commands are not equivalent. The first one truncates the file (file is 0 bytes). The second one truncates the file and then adds a newline (file is 1 byte).
    – Ole Tange
    Jun 22, 2010 at 11:05
5
votes

Here is an excellent collection of tips I came across on digg today.

1
  • useful tips indeed
    – kyku
    Mar 2, 2009 at 20:36
5
votes

I just finished reading Linux tips every geek should know at TuxRadar. Very nice articles for those who know a little bit but want to know more.

5
votes

cut and grep are a very nice way to manage plain text files. You can retrieve whatever you want. cut allows you to "vertical" split the file, and grep allows you to "horizontal" split the file.

The following sentence will split all the lines for ; an only returns the 1 and 3.

$cut -d ';' -f 1,3 hello.txt

With grep (a well known command) you can do the same for lines. The following sentence will ignore the lines which have no interest for you:

$grep error hello.txt

grep can be also for reverse: ignore lines not matching the pattern and also you can use regular expressions.

But the most powerfulness of both are using pipes. For instance:

$grep error hello.txt | cut d ':' -f1,3 | cut -d' ' -f1
4
  • 1
    Neat picking: Once cut fails try awk, once awk fails do perl. Once perl sucks learn python. Mar 6, 2009 at 15:36
  • s/^Neat picking/Nitpicking/ Sep 26, 2009 at 16:15
  • "Horizontal" cut: sed '3,6p;d' hello.txt will print line numbers 3 through six. More concise than piping head and tail together. Sep 26, 2009 at 16:41
  • so much easier when you use the 'grep' built into awk.. awk -F; '/error/ { print $1, $3 }' hello.txt Jan 5, 2011 at 6:38
4
votes

If you want to combine both stdout and stderr in a redirection, try using 2>&1, as in:

make > make.log 2>&1

3
  • 1
    Same, but shorter (and less known): make &> make.log
    – kyku
    Mar 4, 2009 at 18:44
  • @kyku: Nice! This should be it's own comment. Mar 5, 2009 at 1:39
  • 2
    Let it be prize for those reading comments ;-)
    – kyku
    Mar 5, 2009 at 16:29
4
votes

My ISP has a bandwidth cap, but a free-for-all window at starting 2 or 3am. So I schedule huge downloads for that window with at:

$ echo aptitude -d -y dist-upgrade | at 3am
$ echo wget http://example.com/bigfile | at 3am

The thing that originally confused me with at is that it takes the script on stdin, not on the command line.

0
4
votes

Use the watch command to repeat commands and observe results. If it's not supported (as on Solaris systems), try the following (bash):

while [ 1 ] ; do
<cmd>
sleep <n> # n is # of seconds to repeat command
echo "" # meaningful output here can be helpful
        # I like to use ">>>>>>" `date` "<<<<<<<"
done
2
  • To sleep less then a second use usleep Mar 31, 2009 at 7:52
  • Ahh, good point. All my uses for this poll at 2-5 second intervals, but this is helpful as well.
    – bedwyr
    Mar 31, 2009 at 15:01
4
votes
reset

or

stty sane

in case you ruin your terminal by accidentally catting a binary file!

4
votes

You can follow more than one file with tail -f:

tail -f logfile1 logfile2

the updates will be intermingled depending on the order of occurrence:

==> logfile1 <==
event-a
event-b
==> logfile2 <==
event-p
event-q
==> logfile1 <==
event-c

If you want a cleaner display, use watch and omit the -f from tail:

watch tail logfile1 logfile2
2
  • What an awesome tip. I had no idea you could tail more than one file at once. Apr 22, 2010 at 15:15
  • I just figured this out by accident yesterday. I'm in love with this trick. Feb 16, 2012 at 12:13
3
votes

My personal, all time favorite: CLI meta and persistent aliases.

with these aliases, one simple command (val) to define new shell aliases, and have them going forward. concept can be extended to smaller "modes"/domains by additional alias files or aliases.

in ~/.alias.sh (make sure this gets sourced in your shell startup files)

# bash format example
alias sal='. ~/.alias.sh; echo ~/.alias.sh was sourced'
alias val='vi ~/.alias.sh; sal'

or ~/.alias.csh (csh format-- make sure it gets included in your shell start files)

# csh format 
alias sal 'source ~/.alias.csh; echo ~/.alias.csh was sourced'
alias val 'vi ~/.alias.csh ; sal'
1
  • +1 Cool, this looks like something I'll be making use of (with 'vi' replaced with some editor I can use :)
    – Jonik
    Mar 2, 2009 at 22:20
3
votes

My most common ls commands are

ls -lSr
ls -ltr

to sort files in order of increasing size and time respectively (to find the largest or most recent files). Also, if you don't have a colour terminal for some reason, or don't like colours (like me) then ls -F gives you the same sort of metadata as colours: '/' indicates a dir, '*' an executable, etc.

Also, learn to use find and xargs: the power of the command line is cobbling together smaller commands into something more powerful. These 2 commands are indispensible for that!

1
  • I cannot more agree with you - find and xargs are useful. If you are a mac user you need -print0 for find and -0 for xargs, since Mac uses spaces in filenames. Mar 23, 2009 at 22:16
3
votes

here-docs are kind of fun:

cat << EOF > /tmp/file.txt

The ${speed} ${color} ${animal} jumped over the ${structure}

EOF

Or, if using BASH .. fun with here-docs in loops:

cat << EOF > /tmp/file.txt

The ${speed[i]} ${color[i]} ${animal[i]} jumped over the ${structure[i]}

EOF

Handy for generating HTML, PHP, configuration files, or damn near anything else inside of a shell script .. even other shell scripts :)

3
votes

Command-line editing keyboard shortcuts I use in bash:

CTRL-U - Delete text from cursor position back to home. Great for when you mistype a password, and don't remember if your terminal supports DELETE or BACKSPACE.

CTRL-A - Just like HOME key, even when your terminal doesn't send HOME correctly.

CTRL-E - Just like END

ALT-F - Move cursor forward by a word.

ALT-B - Move cursor backward by a word.

5
  • Also Ctrl+Right Arrow (Does the same as ALT-F) Mar 3, 2009 at 9:37
  • That works for me only in Windows. when I try it in an ANSI terminal in putty it doesn't really work... Mar 3, 2009 at 9:56
  • Ctrl-K cut from cursor position to end of line Ctrl-Y paste last cut text, after which you can repeteadly press Alt-Y, which will paste older texts.
    – kyku
    Mar 5, 2009 at 17:17
  • Awesome! I didn't know about the ALT-Y! Mar 8, 2009 at 8:06
  • ^-Y is useful similarly as ^-U. I did not know that ^-U cleans and puts the removed text to your pbcopy. Thank you for sharing the pieces of information! Mar 23, 2009 at 22:12
3
votes

Add this to ~/.bashrc

# expand things like !310:1 after spaces.
bind Space:magic-space

It's a bit scary to use

  • !:0 (0th argument of last command, i.e. the program name)
  • !:2 (2th argument of last command)
  • !! (the whole of last command) (sudo !!)
  • !$ (last argument of last command)
  • !* (all arguments of last command)
  • !ssh (last command starting with ssh)
  • ^chunky^bacon (last command except the first chunky is replaced with bacon)
  • !:gs\chunky\bacon (... all chunky is replaced with bacon)

without checking what's being substituted before running the command.

When you use wildcards like *.txt or globs like hello.{txt,org}, you can check what's being done with echo command beforehand.

echo rm *.bak
rm *.bak

But for stuff like !:0, you don't use echo, because once echo is done, the last command is the echo command. You may have heard of this phenomena as the "observation screw things" principle in quantum mechanics.

Instead, you just add "bind Space:magic-space" in your ~/.bashrc and then, whenever you press space, the stuffs like !:0 is expanded right there.

0
3
votes

I'm surprised no one mentioned bash's built-in fc command (fc stands for Fix Command).

Use it to edit your previous command in an editor (i.e. vim) instead of in the command line, and execute it upon quitting the editor. Pretty handy.

fc [-e ename] [-nlr] [first] [last]
fc -s [pat=rep] [command]

Read more about it here

2
votes

Search in a gzipped file without first unzipping it:

gzcat someFile.gz | grep searchString
2
  • Similarly, bzcat works on bzip2'd files. Quite handy for patches that are distributed bzipped - bzcat patch.bz2 | patch -p1 avoids the need to extract it somewhere temporary.
    – Peter
    Mar 3, 2009 at 1:27
  • 7
    How about zgrep, zless and zxpdf? ;-)
    – Jonas Kölker
    Mar 3, 2009 at 9:57
2
votes

A few useful ones:

rsync -av old_location new_location

will copy a directory structure and preserve permissions and links.

sudo updatedb && locate filename

to find files quickly (requires findutils)

apropos term_or_terms

searches the manpages.

2
votes
alias rm='rm -i'
3
  • 4
    That is a standard no-no. One day, you'll not have the alias set, and then you'll accidentally remove something precious. Mar 3, 2009 at 2:48
  • I use this too, but I don't depend on it.
    – gpojd
    Mar 3, 2009 at 17:02
  • I have a script called rn: mkdir ~/.rm 2>/dev/null; mv "$@" ~/.rm; This makes a wastebasket that in ~/.rm from where you can rescue stuff that should not be removed. When you disk is running full empty the wastebasket: rm -rf ~/.rm
    – Ole Tange
    Jun 22, 2010 at 11:30
2
votes

If you find yourself thinking "oh, I can just write about 15 lines of Perl/Python/whatever to do what I want", first take a look at the coreutils.

2
votes

You can do path substitutions to change dirs by using 2 args with cd:

$ pwd
/foo/bar/blah
$ cd bar bat
/foo/bat/blah
1
  • 1
    What shell? What option enables this? It doesn't work for me in Bash 3. Sep 26, 2009 at 16:48
2
votes

use expect ! This makes scripting interactive tools much easier. For example you can script telnet session, or ftp session.

Do your work on the command line, and then just script it. Here is a crude example, to telnet on a developement board, retrieve a kernel image, and put it into flash

#!/bin/bash
# scripted telnet
IP=$1
IMAGE="platform-AT91SAM9260/images/linuximage"
cp $IMAGE /home/cynove/public_html/
expect -b - <<EndOfTelnet
spawn telnet $IP
expect "login"
send "root\r"
expect "#"
set timeout -1
send "wget -O kimage http://192.168.10.2/~cynove/linuximage\r"
expect "#"
send "ls -al kimage\r"
expect "kimage"
send "flashcp -v kimage /dev/mtd1\r"
expect "Erasing"
expect "#"
send "exit\r"
EndOfTelnet
2
votes

I like the following commands which gives you Google and Google books in terminal

#!/bin/sh

q=$1
w=$2
e=$3
r=$4
t=$5

open "http://www.google.com/search?q=$q+$w+$e+$r+$t&ie=utf-8&oe=utf-8:en-GB:official&client=firefox-a"

and Google books in terminal

#!/bin/sh

q=$1
w=$2
e=$3
r=$4
t=$5

open "http://books.google.com/books?q=$q+$w+$e+$r+$t"

I have the following in .bashrc

alias gb='/Users/Mas/bin/googlebooks'                                                                               

alias g='/Users/Mas/bin/google'

I have permissions 777 for the scripts at /bin/googleScripts/.

Examples

gb Strang null space            // searches books which have the words

g Lordi Hard Rock Eurovision   // Googles websites which have the words

It would be cool to have book titles in terminal so that you do not need an external browser.

Similarly, it would be useful to have Google searches' urls directly in Terminal such that you can process the data in Terminal.

1
  • What's with the variable assignments? Just use the positional parameters directly. Also, I think open is Mac-specific. Sep 26, 2009 at 16:51
2
votes

One of my favorite cli trick is to get out of CLI.

xdg-open .

This opens a GUI file browser with the current directory. That's for Linux. Use "START ." for Windows and "open ." for OS X)

Before I learned to use echo, I was afraid to use rm with wildcards, so I would xdg-open the current folder and then remove files in GUI. I was also afraid of the tar command, another reason to use xdg-open.

What about a way to get back to the CLI world?

Double-click (or triple-click) on the location bar of your GUI file browser and run:

cd "$(xsel)"

that is from here. (or use open-terminal-here with nautilus)

2
votes

Safety at the Command Line

Experienced sysadmins do things that appear lazy or idiosyncratic at first, but save their bacon once in a blue moon.

Don't Type Anything You Can Paste

Prefer copy and paste to typing, whether you're copying from a run book, manual, or just higher in the terminal window. It's all too easy to type the wrong argument, switch, command, filename, etc., especially when you're simultaneously looking at systems, reporting status on a conference call, and trying to figure out the root cause of a problem.

Pasting the command line is a good habit. We should enable it by a) making everything scriptable and b) putting commands in manuals instead of screenshots.

Differentiate Windows

"Oops, wrong shell!" Famous last words. Find some way to separate windows that log in to different environments. Give production environments a different background color, or put them on a different monitor.

Don't Trust the Path

An oldie, but goodie, is to make a shell script called "ls" in your directory. Make it suid root, have it create a suid root copy of /bin/bash in a hidden directory of your own, then delete itself and run the real /bin/ls. All you need to do then is get a naive admin to run "ls -la" in your directory and poof, you've got a root shell.

Real admins never have "." in their paths, exactly to avoid this kind of sneak attack. You shouldn't either. Adding "./" to the front of nearby executables is a good habit to get into.

Don't Move Files. Copy Then Delete

"mv oldname newname" is dangerous. It can destroy two files at once. It's better to do a sequence. Copy the origin file to the destination, check whether it's OK, then delete the original. Better yet, wait until you're totally done with the whole process, then remove the original file. Better yet, make a safe copy of the file you're about to change. The goal is to make everything completely reversible, so you can always get back to a known state.

2
votes

Quick & Simple data integrity verification

using nothing more then bash & md5sum

This can prove to be priceless in terms of debugging trouble when moving binary files over the network... You should embrace this technique as common practice for each copy of valuable data to ensure 100% data integrity.

Setup some test data...

mkdir -p /tmp/mdTest/dir{1,2,3}
for i in `seq 1 3`; do echo $RANDOM > /tmp/mdTest/dir$i/file$i ; done

md5 hash calculation on the test data

cd /tmp/mdTest/
TMPMD5LIST=$(mktemp); (find  -type f -exec md5sum '{}' \;) > $TMPMD5LIST; mv $TMPMD5LIST list.md5sum

data integraty verification from the hash

cd /tmp/mdTest/
md5sum --check list.md5sum
./dir3/file3: OK
./dir1/file1: OK
./dir2/file2: OK

Unit test: Let's break one of the files.

echo $RANDOM >> /tmp/mdTest/dir1/file1
md5sum --check list.md5sum
./dir3/file3: OK
./dir1/file1: FAILED
./dir2/file2: OK
md5sum: WARNING: 1 of 3 computed checksums did NOT match
2
votes

Bash parameter expansions are great:

$ FOO=leading-trailing.txt

$ echo ${FOO#leading-} # remove leading match
trailing.txt

$ echo ${FOO%.txt} # remove trailing match
leading-trailing

$ echo ${FOO/-*/-following} # match a glob and substitute
leading-following.txt

Need to rename a bunch of files? Combine with a for loop:

$ for FILE in file*.txt; do mv -v $FILE ${FILE#file-}; done
file-01.txt -> 01.txt
file-02.txt -> 02.txt
file-03.txt -> 03.txt

Once you have a basic grasp on them you'll be surprised how often they're useful.

2
votes

Sometimes while working in terminal you need to open a file using some GUI application associated with it like for pdf or mp3 files. You don't have to remember the exact name of that command, just use:

gnome-open some-file.pdf

BTW, the shortest alias I use is:

alias o=gnome-open

Very handy.

1
vote

My Favorite And Frequently Used:

List contents of tar.gz file

tar -tzf filename.tar.gz

will match line containing S1 OR S2 OR S3 OR S4

grep 'S1.*S2.*S3.*S4' file

Lists all subdirectories of current directory

ls -d */

Total size of directory
du -sh

find a date:

TIMESTAMP=date '+%y%m%d%H%M'

move process from foreground to background

Ctrl-z and then bg

Entire word to uppercase

echo "word" | awk '{print toupper($0)}'

Checks equality between numbers

x -eq y Check is x is equal to y

x -ne y Check if x is not equal to y

x -gt y Check if x is greater than y

x -lt y Check if x is less than y

Checks equality between strings

x = y Check if x is the same as y

x != y Check if x is not the same as y

-n x Evaluates to true if x is not null

-z x Evaluates to true if x is null

Command Line Parameters for ' test '

-d check if the file is a directory

-e check if the file exists

-f check if the file is a regular file

-g check if the file has SGID permissions

-r check if the file is readable

-s check if the file's size is not 0

-u check if the file has SUID permissions

-w check if the file is writetable

-x check if the file is executable

print first field of the last line"

awk '{ field = $1 }; END{ print field }'

important build in variables

$# Number of command line arguments. Useful to test no. of command line args in shell script.

$* All arguments to shell

$@ Same as above

$- Option supplied to shell

$$ PID of shell

$! PID of last started background process (started with &)

1
  • This might be more useful if you splitted separate tips (like the "build-in variables") in separate answers, with short justification and example for each. And maybe skip those 'test' parameter lists altogether - people can find that kind of stuff in manuals and reference guides.
    – Jonik
    Mar 4, 2009 at 7:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .