Linux Basics - sed - HostAfrica
+27 21 554 3096

Blog

Linux Basics – sed

Linux Basics – sed

sed? Surely you mean ‘said’?

As Linux commands go, this one has a very confusing name. That is, until someone tells you, that, like many Linux commands, it is yet another abbreviation. This time, sed is short for stream editor. Stream editor? Let’s explain briefly. Many editors for text files operate in a sort of point-click-and-save fashion. They cannot edit a real-time stream of data such as a log file. SED, on the other hand, is able to modify text streaming through it in real-time. The sed command is a wonderful utility. The language is very simple, but the documentation is terrible. We asked our friendly Linux guru at Host Africa to explain it to us.

The most important feature of sed

The most used and for daily task, the most useful feature of sed is the ‘s‘ function. The ‘s‘ stands for substitution. The substitute command changes all occurrences of the regular expression into a new value. A simple example, using our ‘sample.txt’ file from the previous Linux Basics blog (see below) is changing “dog” in the “sample.txt” file to “cat” in the “new.txt” file.

First, our sample.txt file:

~$: cat sample.txt 
The quick
brown
fox
jumped over
the lazy
dog
.

Now to use the stream editor (sed) to change ‘dog’ to ‘cat’.

~$ cat sample.txt |sed s/dog/cat/
The quick
brown
fox
jumped over
the lazy
cat
.

Notice the command output the content of the file ‘sample.txt’, but we pipe it into the sed command. We then instruct sed to substitute (s) dog with cat ( s/dog/cat/ ). We can also do this without the cat command as sed is clever enough to read files without assistance from a cat. Also, note that we have not yet modified anything.

~$ sed s/dog/cat/ sample.txt 
The quick
brown
fox
jumped over
the lazy
cat
.

Now we will redirect the output from sed to new.txt.

~$ sed s/dog/cat/ sample.txt >new.txt
~$ cat new.txt 
The quick
brown
fox
jumped over
the lazy
cat
.

# Check content of sample.txt
~$ cat sample.txt 
The quick
brown
fox
jumped over
the lazy
dog
.

Now you can see, sed stood in the data stream between sample.txt and new.txt and changed ‘dog‘ to ‘cat‘. In general, it is better to quote your sed expression like this:

sed 's/dog/cat/'
# SAMPLE
~$ echo "dog"
dog
~$ echo "dog" |sed 's/dog/cat/'
cat
# Let be creative
~$ echo "hounddog" |sed 's/dog/cat/'
houndcat

Another important concept is that sed is line oriented. Suppose you have the input file:

one two three, one two three
four three two one
one hundred

and you used the command

~$ sed 's/one/ONE/' <file

The output would be

ONE two three, one two three
four three two ONE
ONE hundred

Note that this changed “one” to “ONE” once on each line. The first line had “one” twice, but only the first occurrence was changed. That is the default behaviour. If you want something different, you will have to use some of the options that are available.

Let’s return to the command: sed ‘s/one/ONE/’

There are four parts to this substitute command:

s	  Substitute command
/../../	  Delimiter
one	  Regular Expression Pattern Search Pattern
ONE	  Replacement string

The search pattern is on the left-hand side and the replacement string is on the right-hand side. The search pattern may be matched using REGULAR EXPRESSIONS, just as we did with the grep or egrep command.

The character after the s is the delimiter. It is conventionally a slash because this is what edmore, and vi use. It can be anything you want. If you want to change a pathname that contains a slash – say /usr/local/bin to /common/bin – you could use the backslash to quote the slash:

sed 's/\/usr\/local\/bin/\/common\/bin/' 

but it is easier to read if you use an underline instead of a slash as a delimiter:

sed 's_/usr/local/bin_/common/bin_'

Use any delimiter you like, as long as it does not occur in the search string. Always remember that you need 3 delimiters.

Many, many more options we won’t mention here…

The sed command has many more options and is incredibly powerful. As this is a Linux Basics series, we will not delve into the magic of sed any further. One of the best comprehensive guides to sed is by Bruce Barnet at http://www.grymoire.com/Unix/Sed.html.

sedHappy Hosting!

 

Leave a Reply