How to use Linux sed ‘s’ Function
As Linux commands go, this one has a very confusing name. That is, until someone tells you, that, like many Linux commands, it is yet another abbreviation. This time, sed is short for stream editor. Stream editor? Let’s explain briefly. Many editors for text files operate in a sort of point-click-and-save fashion. They cannot edit a real-time stream of data such as a log file. SED, on the other hand, is able to modify text streaming through it in real-time. The sed command is a wonderful utility. The language is very simple, but the documentation is terrible. We asked our friendly Linux guru at HOSTAFRICA to explain it to us.
The most important feature of sed
The most used and for daily task, the most useful feature of sed is the ‘s‘ function. The ‘s‘ stands for substitution. The substitute command changes all occurrences of the regular expression into a new value. A simple example, using our ‘sample.txt’ file from the previous Linux Basics blog (see below) is changing “dog” in the “sample.txt” file to “cat” in the “new.txt” file.
First, our sample.txt file:
~$: cat sample.txt The quick brown fox jumped over the lazy dog .
Now to use the stream editor (sed) to change ‘dog’ to ‘cat’.
~$ cat sample.txt |sed s/dog/cat/ The quick brown fox jumped over the lazy cat .
Notice the command output the content of the file ‘sample.txt’, but we pipe it into the sed command. We then instruct sed to substitute (s) dog with cat ( s/dog/cat/ ). We can also do this without the cat command as sed is clever enough to read files without assistance from a cat. Also, note that we have not yet modified anything.
~$ sed s/dog/cat/ sample.txt The quick brown fox jumped over the lazy cat .
Now we will redirect the output from sed to new.txt.
~$ sed s/dog/cat/ sample.txt >new.txt ~$ cat new.txt The quick brown fox jumped over the lazy cat . # Check content of sample.txt ~$ cat sample.txt The quick brown fox jumped over the lazy dog .
Now you can see, sed stood in the data stream between sample.txt and new.txt and changed ‘dog‘ to ‘cat‘. In general, it is better to quote your sed expression like this:
sed 's/dog/cat/' # SAMPLE ~$ echo "dog" dog ~$ echo "dog" |sed 's/dog/cat/' cat # Let be creative ~$ echo "hounddog" |sed 's/dog/cat/' houndcat
Another important concept is that sed is line oriented. Suppose you have the input file:
one two three, one two three four three two one one hundred
and you used the command
~$ sed 's/one/ONE/' <file
The output would be
ONE two three, one two three four three two ONE ONE hundred
Note that this changed “one” to “ONE” once on each line. The first line had “one” twice, but only the first occurrence was changed. That is the default behaviour. If you want something different, you will have to use some of the options that are available.
Let’s return to the command: sed ‘s/one/ONE/’
There are four parts to this substitute command:
s Substitute command /../../ Delimiter one Regular Expression Pattern Search Pattern ONE Replacement string
The search pattern is on the left-hand side and the replacement string is on the right-hand side. The search pattern may be matched using REGULAR EXPRESSIONS, just as we did with the grep or egrep command.
The character after the s is the delimiter. It is conventionally a slash because this is what ed, more, and vi use. It can be anything you want. If you want to change a pathname that contains a slash – say /usr/local/bin to /common/bin – you could use the backslash to quote the slash:
but it is easier to read if you use an underline instead of a slash as a delimiter:
Use any delimiter you like, as long as it does not occur in the search string. Always remember that you need 3 delimiters.
Many, many more options we won’t mention here…
The sed command has many more options and is incredibly powerful. As this is a Linux Basics series, we will not delve into the magic of sed any further.
One of the best comprehensive guides to sed is by Bruce Barnet at http://www.grymoire.com/Unix/Sed.html.