Shell scripting tutorial

This is a shell scripting tutorial. We use the bash shell as it is standard in most Linux distributions.

MindMap

Hello world

The easiest way to get your feet wet with a programming language is to start with a program that simply outputs a trivial text, the so-called hello-world-example. Here it is for bash:

• create a file named hello in your home directory with the following content:

#!/bin/bash
echo "Hello, world!"

• open a console, enter

cd
chmod +x hello

• now you can execute your file like this:

# ./hello
Hello, world!

• or like this:

# bash hello
Hello, world!

You see - the output of your shell program is the same as if you had entered the commands into a console.

calling commands

In your shell script you can call every command that you can call when opening a console:

echo "This is a directory listing, latest modified files at the bottom:"
ls -ltr
echo "Now calling a browser"
firefox
echo "Continuing with the script"

variables

input

To show you how to deal with variables, we will now write a script that asks for your name and greets you:

echo "what is your name? "
read name
echo "hello $name"

You see that the name is stored in a variable $name. Note the quotation marks " around "hello $name". By using these you say that you want variables to be replaced by their content. If you were to use apostrophes, the name would not be printed, but $name instead.

${}

The ${} operator stands for the variable, there is no difference if you write

echo "$name"

or

echo "${name}"

So what is the sense of this? Imagine you want to echo a string directly, without any blank, after the content of a variable:

echo "if I add the syllable owa to your name it will be ${name}owa"

common mistakes

Note that the variable is called $name, however the correct statement to read it from the keyboard is

read name

It is a common mistake to write

read $name

which means "read a string and store it into the variable whose name is stored in $name"

arrays

How to define an array:

# a=(this is a test)

How to display its members:

# echo ${a[0]}
this
# echo ${a[1]}
is
# echo ${a[2]}
a
# echo ${a[3]}
test

Display the complete array:

# echo ${a[@]}
this is a test

Display its length:

# echo ${#a}
4

parameters

echo "Here are all parameters you called this script with: $@"
echo "Here is parameter 1: $1"
echo "Which parameter do you want to be shown? "
read number
args=("$@")
echo "${args[$number-1]}"

return codes

Every bash script can communicate with the rest of the system by

• sending data to stdout
• sending data to stderr
• delivering a return code

The return code is 0 if everything worked well. You can query it for the most recent command using $?:

$ echo "hello world"; echo $?
hello world
0
$ echo "hello world">/proc/cmdline; echo $?
bash: /proc/cmdline: Permission denied
1

In bash, true is 0 and false is any value but 0. There exist two commands, true and false that deliver true or false, respectively:

$ true; echo $?
0
$ false; echo $?
1

line feeds

Let's look at the following script:

read name
if [ $name = "Thorsten" ]; then echo "I know you"; fi

Instead of a semicolon you can write a line feed like this:

read name
if [ $name = "Thorsten" ]
  then echo "I know you"
fi

And instead of a line feed you can use a semicolon:

read name; if [ $name = "Thorsten" ]; then echo "I know you"; fi

If you want to insert a line feed where you do not need one, e.g. to make the code better readable, you must prepend it with a backslash:

read \
  name
if [ $name = "Thorsten" ]
  then \
    echo "I know you"
fi

storing a command's output

To read a command's output into a variable use $(), backticks or piping.

$()

arch=$(uname -m)
echo "Your system is a $arch system."

backticks

arch=`uname -m`
echo "Your system is a $arch system."

piping

uname -m | while read arch; do echo "Your system is a $arch system."; done

comparison

The advantage of using backticks over $() is that backticks also work in the sh shell. The advantage of using $() over backticks is that you can cascade them. In the example below we use this possibility to get a list of all files installed with rpm on the system:

filelist=$(rpm -ql $(rpm -qa))

You can use the piping approach if you need to cascade in sh, but this is not focus of this bash tutorial.

common mistakes

Usually unexperienced programmers try something like

uname -m | read arch

which does not work. You must embed the read into a while loop.

conditions

The easiest form of a condition in bash is this if example:

echo "what is your name? "
read name
if [ $name = "Thorsten" ]; then echo "I know you"; fi

Now let's look closer at this, why does it work? Why is there a blank needed behind the [ sign? The answer is that [ is just an ordinary command in the shell. It delivers a return code for the expression that follows till the ] sign. To prove this we can write a script:

if true; then echo "the command following if true is being executed"; fi
if false; then echo "this will not be shown"; fi

empty strings

An empty string evaluates to false inside the [ ] operators so it is possible to check if a string result is empty like this:

tweedleburg:~ # result=
tweedleburg:~ # if [ $result ]; then echo success; fi
tweedleburg:~ # result=good
tweedleburg:~ # if [ $result ]; then echo success; fi
success

arithmetic expressions

echo "what is your age? "
read age
if (( $age >= 21 )); then echo "Let's talk about sex."; fi

not equal

To check if a variable is NOT equal to whatever, use !=:

if [ "$LANG" != "C" ]; then echo "please set your system lanugage to C"; fi

common mistakes

Common mistakes are:

• to forget the blank behind/before the [ or ] character
• to forget the blank behind/before the equal sign
• see what does "unary operator expected" mean

Redirections

To redirect the output of a command to a file you have to consider that there are two output streams in UNIX, stdout and stderr.

filling files

To create a file, probably the easiest way is to use cat. The following example writes text into README till a line occurs that only contains the string "EOF":

cat >README<<EOF
This is line 1
This is line 2
This is the last line
EOF

Afterwards, README will contain the 3 lines below the cat command and above the line with EOF.

loops

for loops

Here is an example for a for-loop. It makes a backup of all text files:

for i in *.txt; do cp $i $i.bak; done

The above command takes each .txt file in the current directory, stores it in the variable $i and copies it to $i.bak. So file.txt gets copied to file.txt.bat.

You can also use subsequent numbers as a for loop using the command seq like this:

for i in $(seq 1 1 3); do echo $i; done

Me too! Hope it is working out for you. I was lucky egounh to have one meeting cancelled so I am using that to catch up on things with some much needed desk time - like checking a few blogs :) The rest of the day is fairly full, but at least not stressfull. Enjoy the time to refresh if you get any!!Much love,B

negations

You can negate a result with the ! operator. $? is the last command's return code:

# true
# echo $?
0
# false
# echo $?
1
# ! true
# echo $?
1
# ! false
# echo $?
0

So you get an endless loop out of:

while ! false; do echo hallo; done

The following code checks the file /tmp/success to contain "success". As long as this is not the case it continues checking:

while ! (grep "success" /tmp/success)
do
  sleep 30
done

The following code checks if the file dblog.log exists. As long as this is not the case it tries to download it via ftp:

while ! (test -e dblog.log); do
  ftp -p ftp://user:password@server/tmp/dblog.log >/dev/null
  echo -en "."
  sleep 1
done

sending a process to the background

To send a process to the background, use the ampersand sign (&):

firefox & echo "Firefox has been started"

You see a newline is not needed after the &

forking a process

You can build a process chain using parantheses. This is useful if you want to have two instruction streams being executed in parallel:

(find -iname "helloworld.txt") & (sleep 5; echo "Timeout exceeded, killing process"; kill $!)

functions

To define a function in bash, use a non-keyword and append opening and closing parentheses. Here a function greet is defined and it prints "Hello, world!". Then it is called:

greet()
{
    echo "Hello, world!"
}
greet

react on CTRL_C

The command trap allows you to trap CTRL_C keystrokes so your script will not be aborted

#!/bin/bash

trap shelltrap INT

shelltrap()
{
    echo "You pressed CTRL_C, but I don't let you escape"
}

while true; do read line; done

Note

You can still pause your script by pressing CTRL_Z, send it to the background and kill it there. To catch CTRL_Z, replace INT by TSTP in the above example. To get an overview of all signals that you might be able to trap, open a console and enter

kill -l

thanks for your comment. It would be very helfpul if you put some examples of your expected output (as you mentioned : I want to filter some fields which is important from csv files.)... Please post your input and your expected output. I would definitely help you on this. Thanks.

See also

• bash
• shell
• scripting
• bash operators
• http://en.wikibooks.org/wiki/Bash_Shell_Scripting
• http://ryanstutorials.net/linuxtutorial/scripting.php
• http://wiki.linuxquestions.org/wiki/Bash_tips
• http://wiki.linuxquestions.org/wiki/Bash
• http://linuxconfig.org/Bash_scripting_Tutorial
• http://steve-parker.org/sh/intro.shtml
• http://mywiki.wooledge.org/BashFAQ
• http://mywiki.wooledge.org/BashPitfalls

Share on Facebook