Shell scripting tutorial
This is a shell scripting tutorial. We use the bash shell as it is standard in most Linux distributions.
Contents
- 1 Hello world
- 2 calling commands
- 3 variables
- 4 parameters
- 5 return codes
- 6 line feeds
- 7 reading a command's output stream
- 8 conditions
- 9 Redirections
- 10 filling files
- 11 loops
- 12 sending a process to the background
- 13 forking a process
- 14 awk: read a specific column
- 15 grep: search a string
- 16 sed: replace a string
- 17 tr: replace linebreaks
- 18 See also
Hello world
The easiest way to get your feet wet with a programming language is to start with a program that simply outputs a trivial text, the so-called hello-world-example. Here it is for bash:
- create a file named hello in your home directory with the following content:
#!/bin/bash echo "hello world"
- open a console, enter
cd chmod +x hello
- now you can execute your file like this:
# ./hello hello world
- or like this:
# bash hello hello world
You see - the output of your shell program is the same as if you had entered the commands into a console.
calling commands
In your shell script you can call every command that you can call when opening a console:
echo "This is a directory listing, latest modified files at the bottom:" ls -ltr echo "Now calling a browser" firefox echo "Continuing with the script"
variables
input
To show you how to deal with variables, we will now write a script that asks for your name and greets you:
echo "what is your name? " read name echo "hello $name"
You see that the name is stored in a variable $name. Note the quotation marks " around "hello $name". By using these you say that you want variables to be replaced by their content. If you were to use apostrophes, the name would not be printed, but $name instead.
${}
The ${} operator stands for the variable, there is no difference if you write
echo "$name"
or
echo "${name}"
So what is the sense of this? Imagine a ambigous situation like this
echo "if I add the syllable owa to your name it will be ${name}owa"
common mistakes
Note that the variable is called $name, however the correct statement to read it is
read name
It is a common mistake to write
read $name
which means "read a string and store it into the variable whose name is stored in $name"
parameters
echo "Here are all parameters you called this script with: $@"
echo "Here is parameter 1: $1"
echo "Which parameter do you want to be shown? "
read number
args=("$@")
echo ${args[$number-1]}
return codes
Every bash script can communicate with the rest of the system by
The return code is 0 if everything worked well. You can query it for the most recent command using $?:
bootstick@bootstick:~$ echo "hello world"; echo $? hello world 0 bootstick@bootstick:~$ echo "hello world">/proc/cmdline; echo $? bash: /proc/cmdline: Permission denied 1
In bash, true is 0 and false is any value but 0. There exist two commands, true and false that deliver true or false, respectively:
bootstick@bootstick:~$ true; echo $? 0 bootstick@bootstick:~$ false; echo $? 1
line feeds
Let's look at the following script:
read name if [ $name = "Thorsten" ]; then echo "I know you"; fi
Instead of a semicolon you can write a line feed like this:
read name if [ $name = "Thorsten" ] then echo "I know you" fi
And instead of a line feed you can use a semicolon:
read name; if [ $name = "Thorsten" ]; then echo "I know you"; fi
If you want to insert a line feed where you do not need one, e.g. to make the code better readable, you must prepend it with a backslash:
read \
name
if [ $name = "Thorsten" ]
then \
echo "I know you"
fi
reading a command's output stream
To read a command's output into a variable use $(), backticks or piping.
$()
arch=$(uname -m) echo "Your computer is a $arch computer."
backticks
arch=`uname -m` echo "Your computer is a $arch computer."
piping
uname -m | while read arch; do echo "Your computer is a $arch computer."; done
comparison
The advantage of using backticks over $() is that backticks also work in the sh shell. The advantage of using $() over backticks is that you can cascade them. In the example below we use this possibility to get a list of all files installed with rpm on the system:
filelist=$(rpm -ql $(rpm -qa))
You can use the piping approach if you need to cascade in sh, but this is not focus of this bash tutorial.
common mistakes
Usually unexperienced programmers try something like
uname -m | read arch
which just does not work. You must embed the read into a while loop.
conditions
The easiest form of a condition in bash is this if example:
echo "what is your name? " read name if [ $name = "Thorsten" ]; then echo "I know you"; fi
Now let's look closer at this, why does it work? Why is there a blank needed behind the [ sign? The answer is that [ is just an ordinary command in the shell. It delivers a return code for the expression that follows till the ] sign. To prove this we can write a script:
if true; then echo "the command following if true is being executed"; fi if false; then echo "this will not be shown"; fi
arithmetic expressions
echo "what is your age? " read age if (( $age >= 21 )); then echo "Let's talk about sex."; fi
common mistakes
Common mistakes are:
- to forget the blank behind/before the [ or ] character
- to forget the blank behind/before the equal sign
- see what does "unary operator expected" mean
Redirections
To redirect the output of a command to a file you have to consider that there are two output streams in UNIX, stdout and stderr.
filling files
To create a file, probably the easiest way is to use cat:
cat >README<<EOF This is line 1 This is line 2 This is the last line EOF
loops
for loops
Here is an example for a for-loop:
for i in $(seq 1 1 3); do echo $i; done
while loops
$ while true; do read line; done
sending a process to the background
To send a process to the background, use the ampersand sign (&):
firefox & echo "Firefox has been started"
You see a newline is not needed after the &
forking a process
(find -iname "helloworld.txt") & (sleep 5; echo "Timeout exceeded, killing process"; kill $!)
awk: read a specific column
awk is a program that is installed on almost all Linux distributions. It is a good helper for text stream processing. It can extract columns from a text. Let's imagine you want to use the program vmstat to find out how high the CPU user load was. Here is the output from vmstat:
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------ r b swpd free buff cache si so bi bo in cs us sy id wa st 1 0 0 1304736 190232 885840 0 0 20 8 11 17 1 0 99 0 0
We see the user load is in colum 13, and we only want to print this column. We do it with the following command:
vmstat 5 | awk '{print $13}'
This will print a line from vmstat all 5 seconds and only write the column 13. It looks like this:
# vmstat 5 | awk '{print $13}'
us
1
1
0
1
To store the CPU user load into a variable we use
load=$(vmstat 1 2 | tail -n 1 | awk '{print $13}')
What happens here? First vmstat outputs some data in its first line. The data about CPU load can only be rubbish because it did not have any time to measure it. So we let it output 2 lines and wait 1 second between them ( => vmstat 1 2 ). From this command we only read the last line ( => tail -n 1 ). From this line we only print column 13 ( => awk '{print $13}' ). This output is stored into the variable $load ( => load=$(...) ).
grep: search a string
grep is a program that is installed on almost all Linux distributions. It is a good helper for text stream processing. It can extract lines that contain a string or match a regex pattern. Let's imagine you want all external links from www.linuxintro.org's main page:
wget -O linuxintro.txt http://www.linuxintro.org grep "http:" linuxintro.txt
sed: replace a string
sed is a program that is installed on almost all Linux distributions. It is a good helper for text stream processing. It can replace a string by another one. Let's imagine you want to print your distribution's name, but lsd_release outputs too much:
# lsb_release -d Description: openSUSE 12.1 (x86_64)
You want to remove this string "Description" so you replace it by nothing:
lsb_release -d | sed "s/Description:\t//" openSUSE 12.1 (x86_64)
tr: replace linebreaks
sed is a program that is installed on almost all Linux distributions. It is a good helper for text stream processing. It can replace a character by another one, even over line breaks. For example here is how you remove all empty lines from your processor information:
# cat /proc/cpuinfo | while read a; do ar=$(echo -n $a|tr '\
' ';'); if [ "$ar" <> ";" ]; then echo "$ar"; fi; done
