Normally we use a interactive text editor, like vim. The command sed is one of the most commonly used command line editor in Linux world. sed stands for the stream editor, it uses the roles supplied to edit a stream of data on the fly.
Note: Be careful that sed will break the softlink and create a file with the same name! for example:
1 | ln -s /tmp/source.txt /tmp/link.txt |
then the softlink is gone, a new file named link.txt is created instead.
so use readlink first to get resolved symbolic links, then use sed on it.
First you need to understand how sed works with text:
- Reads one data line at a time from the input
- Matches that data with the supplied editor commands
- Changes data in the stream as specified in the commands
- Outputs the new data to STDOUT
2个对于输出很有帮助的flags, 可以加在其他功能后面:
1 | # -n: quiet |
Substitution
I have a text file sedtxt:
1 | Using Java print hello world and hello tree. |
If I want to substite all hello with hi in-place:
1 | sed -i -e 's/hello/hi/g' sedtxt |
-i: does in-place substitution in file sedtxt, create backup file automatically by using -i.bak
-e: followed by commands
s/x/y/flags: substitute option, / is the delimiter, can be other chars; g represents that replace in all occurrences (global).
Note that if not set g, it will replace first occurrence in each line, what if I want to replace the first occurrence in a file, the workaround could be limit the scaning range:
1 | sed '0,/Apple/{s/Apple/Banana/}' input_filename |
the explain see here
If I want to substitute the second hello with goodbye in each line:
1 | sed -e 's/hello/goodbye/2' sedtxt |
1 | Using Java print hello world and goodbye tree. |
If I want to substitute hello with hi and Java with Python:
1 | sed -e 's/hello/hi/g' -e 's/Java/Pyhton/g' sedtxt |
1 | Using Pyhton print hi world and hi tree. |
If I want to print only the matching lines, convenient for debugging:
1 | sed -n -e 's/hello/hi/gp' sedtxt |
1 | Using Java print hi world and hi tree. |
-n: quiet output
p: substitute flag to print matching line
Note, combine with
grepto debug is good
Using Address
The sed editor assigns the first line in the text stream as line number 1 and continues sequentially for each new line.
only replace 2rd line:
1 | sed -e '2s/hello/hi/g' sedtxt |
1 | Using Java print hello world and hello tree. |
range substitution, '1,$s/hello/hi/g' means from top to bottom.
1 | sed -e '2,3s/hello/hi/g' sedtxt |
1 | Using Java print hello world and hello tree. |
can also use text pattern to filter lines, this will apply one line contains print word.
1 | sed -e '/print/s#and#or#' sedtxt |
1 | Using Java print hello world or hello tree. |
Deletion
Delete consecutive lines after match
I have a text file seddel:
1 | The 1st line is 1 |
Delete line 2 to end:
1 | sed -e '2,$d' seddel |
1 | The 1st line is 1 |
can also use pattern matching, delete 3rd
1 | sed -e '/3rd/d' seddel |
1 | The 1st line is 1 |
You can combine 2 address syntax, this will start from first line and until match 3rd, replace 3rd in the range with NAN:
1 | sed -e '1,/3rd/{s/3rd/NAN/g}' seddel |
1 | The 1st line is 1 |
Delete commented and empty line
1 | sed -e '/^#/d; /^$/d' <file> |
Insertion and Appending
The insert command (i) adds a new line before the specified line.
The append command (a) adds a new line after the specified line.
I have a text file sedins:
1 | The 1st line is 1 |
Insert at first line:
1 | sed -e '1iNew line coming!' sedins |
1 | New line coming! |
Append at 3rd line:
1 | sed -e '2aNew line coming!' sedins |
1 | The 1st line is 1 |
Insert with white spaces
For example, When developing non-root, I want to add runAsUser: 1000 right after securityContext: with correct alignment:
1 | sed -i -e '/securityContext/a\ runAsUser: 1000' xxx.yml |
Only to escape the first space. sed can automatically recognize the rest of the spaces.
1 | ... |
Changing
The change command allows you to change the contents of an entire line of text in the data stream.
I have a text file sedch:
1 | The 1st line is 1 |
change the second line:
1 | sed -e '2cNONE' sedch |
1 | The 1st line is 1 |
Transforming chars
The transform command (y) is the only sed editor command that operates on a single character.
I have a text file sedtrans:
1 | The 1st line is 1 |
The transform command performs a one-to-one mapping of the inchars and the outchars values.
1 | sed -e 'y/1234/5678/' sedtrans |
1 | The 5st line is 5 |
The transform command is a global command; that is, it performs the transformation on any character found in the text line automatically, without regard to the occurrence. You can’t limit the transformation to a specific occurrence of the character.
sed files in directory and subdirectories recursively
Actually we can find all files by find then exec sed, see this post:
1 | find <dir> -type f -name "*sh" -exec sed -i -e "s|${old}|${new}|g" {} \; |
Note that
-type fis necessary, otherwise will pass directory name tosed.