Piping

The most fascinating thing about Unix is how the following works together:

• the way Unix treats streams
• the paradigm "everything is a file"
• the paradigm "do one thing and do it well"

Let's start with the most basic program: cat. cat inputs a "stream" and outputs a "stream", nothing else. Find it out like this:

poochy:~ # cat
Hello, I am typing a line
Hello, I am typing a line

After typing cat with no parameters, you can input a line (followed by ENTER). cat does nothing but output the line again. You end your input with CTRL_D. Standard for your input is the keyboard, standard for a program's output is the console.

How can this be useful?

This can be extremely useful. E.g. take the paradigm "everything is a file". So you can replace your input stream by a file using a parameter:

cat readme.txt

outputs the content of readme.txt to the console. So you can display a file.

Or you can redirect cat's output stream to a file using the > operator:

cat > myfile.txt

Allows you to (over)write directly the file myfile.txt.

Or you can redirect the output of cat to the input of another command using the | (pipe) operator:

cat myfile.txt | cat

is equivalent to cat myfile.txt.

How can this pipe be useful?

There is a command to search a string in a stream (yes, or in a file, because everything is a file), called grep.

grep needle

takes an input stream and echos only those lines containing the string "needle":

poochy:/usr # grep needle
haystack
hay
more hay
even more hay
needle
needle
again hay
some more hay

Now if we combine cat and grep, we get

cat * | grep needle

"*" stands for "all files". This command searches in all files for the string "needle" and outputs the files that contain it. Just FYI, this is equivalent to

grep "needle" *

because, as said, you can hand over files to grep.

See also

• shell scripting tutorial