I would like to open a discussion that would accumulate your Linux command line (CLI) best practices and tips.
I've searched for such a discussion to share the below comment but haven't found one, hence this post.
I hope we all could learn from this.
You are welcome to share your Bash tips, grep, sed, AWK, /proc and all other related Linux/Unix system administration, shell programming best practices for the benefit of us all.
$_ is an environment variable for the last argument from the previous command
So if I create a new directory
$ mkdir ~/newdir/
To enter I simply go
$ cd $_
This is handy for complicated and large texts such as URLs, directories, long file names etc.
You can also refer to each argument in the previous command using !:{number}
$ echo !:0 !:1 !:2
Note that bash will expand this before you execute it (to see this press up to go through your history).
$ touch one two three
$ ls !:1 !:2 !:3
Unlike $_ which is an environment variable, this will expand to 'ls one two three', perform the action and print the command to the shell. This method is a lot harder (in my opinion) to use than $_ which I use much more frequently.
Note: you can also use !$ instead of $_ but the former will expand
There was an answer about using pushd/popd to remember directories. If you want to temporarily visit some directory, you can save typing by using cd - command, like this:
/home/eugene $ cd /etc
/etc $ cd -
/home/eugene $
Also, one of my favorite commands is xargs. I use it very often with find, but it can be also handy in other situations. For example, to find which command line arguments were used for starting some process, you can use the following command on linux:
$ xargs -0 echo < /proc/[PID]/cmdline
In some cases (especially when working with source code) ack is more handy than grep because it automatically searches recursively and ignores backup files and version control directories (like .svn, .hg). No more typing long command lines like find . -name \*.c | xargs grep 'frobnicate'.
(cd test; dosomething) leaves you in the directory you started from.
Commented
Sep 26, 2009 at 16:13
A nice one I've seen today from a friend.
Clean the log of the application (for next launch and co.)
> /var/log/appname.log
(Note the the > is part of the command).
This is the same as doing:
echo '' > /var/log/appname.log
Here is an excellent collection of tips I came across on digg today.
I just finished reading Linux tips every geek should know at TuxRadar. Very nice articles for those who know a little bit but want to know more.
cut and grep are a very nice way to manage plain text files. You can retrieve whatever you want. cut allows you to "vertical" split the file, and grep allows you to "horizontal" split the file.
The following sentence will split all the lines for ; an only returns the 1 and 3.
$cut -d ';' -f 1,3 hello.txt
With grep (a well known command) you can do the same for lines. The following sentence will ignore the lines which have no interest for you:
$grep error hello.txt
grep can be also for reverse: ignore lines not matching the pattern and also you can use regular expressions.
But the most powerfulness of both are using pipes. For instance:
$grep error hello.txt | cut d ':' -f1,3 | cut -d' ' -f1
cut: sed '3,6p;d' hello.txt will print line numbers 3 through six. More concise than piping head and tail together.
Commented
Sep 26, 2009 at 16:41
If you want to combine both stdout and stderr in a redirection, try using 2>&1, as in:
make > make.log 2>&1
My ISP has a bandwidth cap, but a free-for-all window at starting 2 or 3am. So I schedule huge downloads for that window with at:
$ echo aptitude -d -y dist-upgrade | at 3am
$ echo wget http://example.com/bigfile | at 3am
The thing that originally confused me with at is that it takes the script on stdin, not on the command line.
Use the watch command to repeat commands and observe results. If it's not supported (as on Solaris systems), try the following (bash):
while [ 1 ] ; do
<cmd>
sleep <n> # n is # of seconds to repeat command
echo "" # meaningful output here can be helpful
# I like to use ">>>>>>" `date` "<<<<<<<"
done
reset
or
stty sane
in case you ruin your terminal by accidentally catting a binary file!
You can follow more than one file with tail -f:
tail -f logfile1 logfile2
the updates will be intermingled depending on the order of occurrence:
==> logfile1 <==
event-a
event-b
==> logfile2 <==
event-p
event-q
==> logfile1 <==
event-c
If you want a cleaner display, use watch and omit the -f from tail:
watch tail logfile1 logfile2
tail more than one file at once.
Commented
Apr 22, 2010 at 15:15
My personal, all time favorite: CLI meta and persistent aliases.
with these aliases, one simple command (val) to define new shell aliases, and have them going forward. concept can be extended to smaller "modes"/domains by additional alias files or aliases.
in ~/.alias.sh (make sure this gets sourced in your shell startup files)
# bash format example
alias sal='. ~/.alias.sh; echo ~/.alias.sh was sourced'
alias val='vi ~/.alias.sh; sal'
or ~/.alias.csh (csh format-- make sure it gets included in your shell start files)
# csh format
alias sal 'source ~/.alias.csh; echo ~/.alias.csh was sourced'
alias val 'vi ~/.alias.csh ; sal'
My most common ls commands are
ls -lSr
ls -ltr
to sort files in order of increasing size and time respectively (to find the largest or most recent files). Also, if you don't have a colour terminal for some reason, or don't like colours (like me) then ls -F gives you the same sort of metadata as colours: '/' indicates a dir, '*' an executable, etc.
Also, learn to use find and xargs: the power of the command line is cobbling together smaller commands into something more powerful. These 2 commands are indispensible for that!
here-docs are kind of fun:
cat << EOF > /tmp/file.txt
The ${speed} ${color} ${animal} jumped over the ${structure}
EOF
Or, if using BASH .. fun with here-docs in loops:
cat << EOF > /tmp/file.txt
The ${speed[i]} ${color[i]} ${animal[i]} jumped over the ${structure[i]}
EOF
Handy for generating HTML, PHP, configuration files, or damn near anything else inside of a shell script .. even other shell scripts :)
Command-line editing keyboard shortcuts I use in bash:
CTRL-U - Delete text from cursor position back to home. Great for when you mistype a password, and don't remember if your terminal supports DELETE or BACKSPACE.
CTRL-A - Just like HOME key, even when your terminal doesn't send HOME correctly.
CTRL-E - Just like END
ALT-F - Move cursor forward by a word.
ALT-B - Move cursor backward by a word.
Add this to ~/.bashrc
# expand things like !310:1 after spaces.
bind Space:magic-space
It's a bit scary to use
without checking what's being substituted before running the command.
When you use wildcards like *.txt or globs like hello.{txt,org}, you can check what's being done with echo command beforehand.
echo rm *.bak
rm *.bak
But for stuff like !:0, you don't use echo, because once echo is done, the last command is the echo command. You may have heard of this phenomena as the "observation screw things" principle in quantum mechanics.
Instead, you just add "bind Space:magic-space" in your ~/.bashrc and then, whenever you press space, the stuffs like !:0 is expanded right there.
I'm surprised no one mentioned bash's built-in fc command (fc stands for Fix Command).
Use it to edit your previous command in an editor (i.e. vim) instead of in the command line, and execute it upon quitting the editor. Pretty handy.
fc [-e ename] [-nlr] [first] [last]
fc -s [pat=rep] [command]
Read more about it here
Search in a gzipped file without first unzipping it:
gzcat someFile.gz | grep searchString
A few useful ones:
rsync -av old_location new_location
will copy a directory structure and preserve permissions and links.
sudo updatedb && locate filename
to find files quickly (requires findutils)
apropos term_or_terms
searches the manpages.
alias rm='rm -i'
rn: mkdir ~/.rm 2>/dev/null; mv "$@" ~/.rm; This makes a wastebasket that in ~/.rm from where you can rescue stuff that should not be removed. When you disk is running full empty the wastebasket: rm -rf ~/.rm
Commented
Jun 22, 2010 at 11:30
If you find yourself thinking "oh, I can just write about 15 lines of Perl/Python/whatever to do what I want", first take a look at the coreutils.
You can do path substitutions to change dirs by using 2 args with cd:
$ pwd
/foo/bar/blah
$ cd bar bat
/foo/bat/blah
use expect ! This makes scripting interactive tools much easier. For example you can script telnet session, or ftp session.
Do your work on the command line, and then just script it. Here is a crude example, to telnet on a developement board, retrieve a kernel image, and put it into flash
#!/bin/bash
# scripted telnet
IP=$1
IMAGE="platform-AT91SAM9260/images/linuximage"
cp $IMAGE /home/cynove/public_html/
expect -b - <<EndOfTelnet
spawn telnet $IP
expect "login"
send "root\r"
expect "#"
set timeout -1
send "wget -O kimage http://192.168.10.2/~cynove/linuximage\r"
expect "#"
send "ls -al kimage\r"
expect "kimage"
send "flashcp -v kimage /dev/mtd1\r"
expect "Erasing"
expect "#"
send "exit\r"
EndOfTelnet
I like the following commands which gives you Google and Google books in terminal
#!/bin/sh
q=$1
w=$2
e=$3
r=$4
t=$5
open "http://www.google.com/search?q=$q+$w+$e+$r+$t&ie=utf-8&oe=utf-8:en-GB:official&client=firefox-a"
and Google books in terminal
#!/bin/sh
q=$1
w=$2
e=$3
r=$4
t=$5
open "http://books.google.com/books?q=$q+$w+$e+$r+$t"
I have the following in .bashrc
alias gb='/Users/Mas/bin/googlebooks'
alias g='/Users/Mas/bin/google'
I have permissions 777 for the scripts at /bin/googleScripts/.
Examples
gb Strang null space // searches books which have the words
g Lordi Hard Rock Eurovision // Googles websites which have the words
It would be cool to have book titles in terminal so that you do not need an external browser.
Similarly, it would be useful to have Google searches' urls directly in Terminal such that you can process the data in Terminal.
open is Mac-specific.
Commented
Sep 26, 2009 at 16:51
One of my favorite cli trick is to get out of CLI.
xdg-open .
This opens a GUI file browser with the current directory. That's for Linux. Use "START ." for Windows and "open ." for OS X)
Before I learned to use echo, I was afraid to use rm with wildcards, so I would xdg-open the current folder and then remove files in GUI. I was also afraid of the tar command, another reason to use xdg-open.
What about a way to get back to the CLI world?
Double-click (or triple-click) on the location bar of your GUI file browser and run:
cd "$(xsel)"
that is from here. (or use open-terminal-here with nautilus)
Safety at the Command Line
Experienced sysadmins do things that appear lazy or idiosyncratic at first, but save their bacon once in a blue moon.
Don't Type Anything You Can Paste
Prefer copy and paste to typing, whether you're copying from a run book, manual, or just higher in the terminal window. It's all too easy to type the wrong argument, switch, command, filename, etc., especially when you're simultaneously looking at systems, reporting status on a conference call, and trying to figure out the root cause of a problem.
Pasting the command line is a good habit. We should enable it by a) making everything scriptable and b) putting commands in manuals instead of screenshots.
Differentiate Windows
"Oops, wrong shell!" Famous last words. Find some way to separate windows that log in to different environments. Give production environments a different background color, or put them on a different monitor.
Don't Trust the Path
An oldie, but goodie, is to make a shell script called "ls" in your directory. Make it suid root, have it create a suid root copy of /bin/bash in a hidden directory of your own, then delete itself and run the real /bin/ls. All you need to do then is get a naive admin to run "ls -la" in your directory and poof, you've got a root shell.
Real admins never have "." in their paths, exactly to avoid this kind of sneak attack. You shouldn't either. Adding "./" to the front of nearby executables is a good habit to get into.
Don't Move Files. Copy Then Delete
"mv oldname newname" is dangerous. It can destroy two files at once. It's better to do a sequence. Copy the origin file to the destination, check whether it's OK, then delete the original. Better yet, wait until you're totally done with the whole process, then remove the original file. Better yet, make a safe copy of the file you're about to change. The goal is to make everything completely reversible, so you can always get back to a known state.
using nothing more then bash & md5sum
This can prove to be priceless in terms of debugging trouble when moving binary files over the network... You should embrace this technique as common practice for each copy of valuable data to ensure 100% data integrity.
mkdir -p /tmp/mdTest/dir{1,2,3}
for i in `seq 1 3`; do echo $RANDOM > /tmp/mdTest/dir$i/file$i ; done
cd /tmp/mdTest/
TMPMD5LIST=$(mktemp); (find -type f -exec md5sum '{}' \;) > $TMPMD5LIST; mv $TMPMD5LIST list.md5sum
cd /tmp/mdTest/
md5sum --check list.md5sum
./dir3/file3: OK
./dir1/file1: OK
./dir2/file2: OK
echo $RANDOM >> /tmp/mdTest/dir1/file1
md5sum --check list.md5sum
./dir3/file3: OK
./dir1/file1: FAILED
./dir2/file2: OK
md5sum: WARNING: 1 of 3 computed checksums did NOT match
Bash parameter expansions are great:
$ FOO=leading-trailing.txt
$ echo ${FOO#leading-} # remove leading match
trailing.txt
$ echo ${FOO%.txt} # remove trailing match
leading-trailing
$ echo ${FOO/-*/-following} # match a glob and substitute
leading-following.txt
Need to rename a bunch of files? Combine with a for loop:
$ for FILE in file*.txt; do mv -v $FILE ${FILE#file-}; done
file-01.txt -> 01.txt
file-02.txt -> 02.txt
file-03.txt -> 03.txt
Once you have a basic grasp on them you'll be surprised how often they're useful.
Sometimes while working in terminal you need to open a file using some GUI application associated with it like for pdf or mp3 files. You don't have to remember the exact name of that command, just use:
gnome-open some-file.pdf
BTW, the shortest alias I use is:
alias o=gnome-open
Very handy.
My Favorite And Frequently Used:
List contents of tar.gz file
tar -tzf filename.tar.gz
will match line containing S1 OR S2 OR S3 OR S4
grep 'S1.*S2.*S3.*S4' file
Lists all subdirectories of current directory
ls -d */
Total size of directory
du -sh
find a date:
TIMESTAMP=date '+%y%m%d%H%M'
move process from foreground to background
Ctrl-z and then bg
Entire word to uppercase
echo "word" | awk '{print toupper($0)}'
Checks equality between numbers
x -eq y Check is x is equal to y
x -ne y Check if x is not equal to y
x -gt y Check if x is greater than y
x -lt y Check if x is less than y
Checks equality between strings
x = y Check if x is the same as y
x != y Check if x is not the same as y
-n x Evaluates to true if x is not null
-z x Evaluates to true if x is null
Command Line Parameters for ' test '
-d check if the file is a directory
-e check if the file exists
-f check if the file is a regular file
-g check if the file has SGID permissions
-r check if the file is readable
-s check if the file's size is not 0
-u check if the file has SUID permissions
-w check if the file is writetable
-x check if the file is executable
print first field of the last line"
awk '{ field = $1 }; END{ print field }'
important build in variables
$# Number of command line arguments. Useful to test no. of command line args in shell script.
$* All arguments to shell
$@ Same as above
$- Option supplied to shell
$$ PID of shell
$! PID of last started background process (started with &)