linux命令学习-sed命令

1 sed命令介绍

参考链接：https://wangchujiang.com/linux-command/c/sed.html
sed是一个功能强大的流式文本编辑器。主要用于对文本流进行修改、替换、删除等操作。它可以读取文件或标准输入，进行各种编辑操作，然后将结果输出到标准输出或文件中。具体的流程是：

1. 把当前处理的行存储在临时缓冲区中，称为“模式空间”（pattern space）【sed是行处理的编辑器】
2. 用sed命令处理缓冲区中的内容;
3. 处理完成后，把缓冲区的内容送往屏幕,也可以重定向到文件中，注意，原文件内容本身没有变化。
4. 接着处理下一行，这样不断重复，直到文件末尾
   sed主要用来自动编辑一个或多个文件；简化对文件的反复操作；编写转换程序等。

1.1 命令格式

sed [options] pattern file(s)

options

-e<script>或--expression=<script>：以选项中的指定的script来处理输入的文本文件；
-f<script文件>或--file=<script文件>：以选项中指定的script文件来处理输入的文本文件；
-h或--help：显示帮助；
-n或--quiet或——silent：仅显示script处理后的结果；
-V或--version：显示版本信息。

pattern

sed命令

a\ # 在当前行下面插入文本。
i\ # 在当前行上面插入文本。
c\ # 把选定的行改为新的文本。
d # 删除，删除选择的行。
D # 删除模板块的第一行。
s # 替换指定字符
h # 拷贝模板块的内容到内存中的缓冲区。
H # 追加模板块的内容到内存中的缓冲区。
g # 获得内存缓冲区的内容，并替代当前模板块中的文本。
G # 获得内存缓冲区的内容，并追加到当前模板块文本的后面。
l # 列表不能打印字符的清单。
n # 读取下一个输入行，用下一个命令处理新的行而不是用第一个命令。
N # 追加下一个输入行到模板块后面并在二者间嵌入一个新行，改变当前行号码。
p # 打印模板块的行。
P # (大写) 打印模板块的第一行。
q # 退出Sed。
b lable # 分支到脚本中带有标记的地方，如果分支不存在则分支到脚本的末尾。
r file # 从file中读行。
t label # if分支，从最后一行开始，条件一旦满足或者T，t命令，将导致分支到带有标号的命令处，或者到脚本的末尾。
T label # 错误分支，从最后一行开始，一旦发生错误或者T，t命令，将导致分支到带有标号的命令处，或者到脚本的末尾。
w file # 写并追加模板块到file末尾。  
W file # 写并追加模板块的第一行到file末尾。  
! # 表示后面的命令对所有没有被选定的行发生作用。  
= # 打印当前行号码。  
# # 把注释扩展到下一个换行符以前。  

sed替换标记

g # 表示行内全面替换。  
p # 表示打印行。  
w # 表示把行写入一个文件。  
x # 表示互换模板块中的文本和缓冲区中的文本。  
y # 表示把一个字符翻译为另外的字符（但是不用于正则表达式）
\1 # 子串匹配标记
& # 已匹配字符串标记

sed元字符集

^ # 匹配行开始，如：/^sed/匹配所有以sed开头的行。
$ # 匹配行结束，如：/sed$/匹配所有以sed结尾的行。
. # 匹配一个非换行符的任意字符，如：/s.d/匹配s后接一个任意字符，最后是d。
* # 匹配0个或多个字符，如：/*sed/匹配所有模板是一个或多个空格后紧跟sed的行。
[] # 匹配一个指定范围内的字符，如/[sS]ed/匹配sed和Sed。  
[^] # 匹配一个不在指定范围内的字符，如：/[^A-RT-Z]ed/匹配不包含A-R和T-Z的一个字母开头，紧跟ed的行。
\(..\) # 匹配子串，保存匹配的字符，如s/\(love\)able/\1rs，loveable被替换成lovers。
& # 保存搜索字符用来替换其他字符，如s/love/ **&** /，love这成 **love** 。
\< # 匹配单词的开始，如:/\<love/匹配包含以love开头的单词的行。
\> # 匹配单词的结束，如/love\>/匹配包含以love结尾的单词的行。
x\{m\} # 重复字符x，m次，如：/0\{5\}/匹配包含5个0的行。
x\{m,\} # 重复字符x，至少m次，如：/0\{5,\}/匹配至少有5个0的行。
x\{m,n\} # 重复字符x，至少m次，不多于n次，如：/0\{5,10\}/匹配5~10个0的行。  

file

文件：指定待处理的文本文件列表。

2 sed用法示例

2.1 文件准备

sample.txt

1. Hello, World!
2. This is a sample text file for sed command practice.
3. It contains multiple lines of text.
4. Sed stands for Stream Editor.
5. You can use sed to replace text.
6. For example, replacing 'text' with 'TEXT'.
7. This file includes some sample data.
8. Let's see how sed works with line numbers.
9. Line 9 is here just for testing.
10. This line has the word example.
11. Example lines can be very useful.
12. Sometimes, you may want to delete lines.
13. Deleting empty lines is a common task.
14. Regular expressions are powerful in sed.
15. You can also append or insert lines.
16. How about modifying the first line?
17. This is the 17th line of the text.
18. Line 18 contains special characters: !@#$
19. The end of this file is near.
20. Thank you for practicing with sed!

2.2 替换操作：s命令

sed 's/text/TEXT/' sample.txt # 将text替换为TEXT
输出如下：
1. Hello, World!
2. This is a sample TEXT file for sed command practice.
3. It contains multiple lines of TEXT.
4. Sed stands for Stream Editor.
5. You can use sed to replace TEXT.
6. For example, replacing 'TEXT' with 'TEXT'.
7. This file includes some sample data.
8. Let's see how sed works with line numbers.
9. Line 9 is here just for testing.
10. This line has the word example.
11. Example lines can be very useful.
12. Sometimes, you may want to delete lines.
13. Deleting empty lines is a common task.
14. Regular expressions are powerful in sed.
15. You can also append or insert lines.
16. How about modifying the first line?
17. This is the 17th line of the TEXT.
18. Line 18 contains special characters: !@#$
19. The end of this file is near.
20. Thank you for practicing with sed!sed 's/text/TEXT/g' sample.txt # 将text替换为TEXT，并且全面替换
因为sed是以行为单位，所以/g的全面替换是一行中的全面替换
输出如下：
1. Hello, World!
2. This is a sample TEXT file for sed command practice.
3. It contains multiple lines of TEXT.
4. Sed stands for Stream Editor.
5. You can use sed to replace TEXT.
6. For example, replacing 'TEXT' with 'TEXT'.
7. This file includes some sample data.
8. Let's see how sed works with line numbers.
9. Line 9 is here just for testing.
10. This line has the word example.
11. Example lines can be very useful.
12. Sometimes, you may want to delete lines.
13. Deleting empty lines is a common task.
14. Regular expressions are powerful in sed.
15. You can also append or insert lines.
16. How about modifying the first line?
17. This is the 17th line of the TEXT.
18. Line 18 contains special characters: !@#$
19. The end of this file is near.
20. Thank you for practicing with sed!sed -i 's/text/TEXT/g' sample.txt # 将text替换为TEXT，并且全面替换，无输出，-i是直接替换文件中的内容# /g的替换测试可以看看
echo sksksksksksk | sed 's/sk/SK/'
输出：SKskskskskskecho sksksksksksk | sed 's/sk/SK/g'
输出：SKSKSKSKSKSKecho sksksksksksk | sed 's/sk/SK/2g'
输出：skSKSKSKSKSKecho sksksksksksk | sed 's/sk/SK/3g'
输出：skskSKSKSKSKecho sksksksksksk | sed 's/sk/SK/4g'
输出：skskskSKSKSK

在以上命令中，以/为定界符，实际上定界符是任意的，也可适用:或者|或者其他，比如：

sed 's:test:TEXT:g'
sed 's|test|TEXT|g'

2.3 删除操作：d命令

sed '/^$/d' sample.txt #删除空白行
输出【注意，最后一行的空白行没了】：
1. Hello, World!
2. This is a sample TEXT file for sed command practice.
3. It contains multiple lines of TEXT.
4. Sed stands for Stream Editor.
5. You can use sed to replace TEXT.
6. For example, replacing 'TEXT' with 'TEXT'.
7. This file includes some sample data.
8. Let's see how sed works with line numbers.
9. Line 9 is here just for testing.
10. This line has the word example.
11. Example lines can be very useful.
12. Sometimes, you may want to delete lines.
13. Deleting empty lines is a common task.
14. Regular expressions are powerful in sed.
15. You can also append or insert lines.
16. How about modifying the first line?
17. This is the 17th line of the TEXT.
18. Line 18 contains special characters: !@#$
19. The end of this file is near.
20. Thank you for practicing with sed!
sed '2d' sample.txt #删除第二行
输出：
1. Hello, World!
3. It contains multiple lines of TEXT.
4. Sed stands for Stream Editor.
5. You can use sed to replace TEXT.
6. For example, replacing 'TEXT' with 'TEXT'.
7. This file includes some sample data.
8. Let's see how sed works with line numbers.
9. Line 9 is here just for testing.
10. This line has the word example.
11. Example lines can be very useful.
12. Sometimes, you may want to delete lines.
13. Deleting empty lines is a common task.
14. Regular expressions are powerful in sed.
15. You can also append or insert lines.
16. How about modifying the first line?
17. This is the 17th line of the TEXT.
18. Line 18 contains special characters: !@#$
19. The end of this file is near.
20. Thank you for practicing with sed!sed '2,$d' sample.txt #输出第二行到末尾行的所有行
输出：
1. Hello, World!
sed '$d' sample.txt #删除末尾行
输出：
1. Hello, World!
2. This is a sample TEXT file for sed command practice.
3. It contains multiple lines of TEXT.
4. Sed stands for Stream Editor.
5. You can use sed to replace TEXT.
6. For example, replacing 'TEXT' with 'TEXT'.
7. This file includes some sample data.
8. Let's see how sed works with line numbers.
9. Line 9 is here just for testing.
10. This line has the word example.
11. Example lines can be very useful.
12. Sometimes, you may want to delete lines.
13. Deleting empty lines is a common task.
14. Regular expressions are powerful in sed.
15. You can also append or insert lines.
16. How about modifying the first line?
17. This is the 17th line of the TEXT.
18. Line 18 contains special characters: !@#$
19. The end of this file is near.
20. Thank you for practicing with sed!
sed '/^1/d' sample.txt #删除所有1开头的行
输出：
2. This is a sample TEXT file for sed command practice.
3. It contains multiple lines of TEXT.
4. Sed stands for Stream Editor.
5. You can use sed to replace TEXT.
6. For example, replacing 'TEXT' with 'TEXT'.
7. This file includes some sample data.
8. Let's see how sed works with line numbers.
9. Line 9 is here just for testing.
20. Thank you for practicing with sed!

2.4 已匹配字符串标记&

# &表达是\w\+匹配到的单词，属于如果匹配到了this==》输出就变成了[this]
echo this is a test line | sed 's/\w\+/[&]/g'
输出：[this] [is] [a] [test] [line]echo thisisa test line | sed 's/\w\+/[&]/g'
输出：[thisisa] [test] [line]#所有以192.168.0.1开头的行都会被替换成它自已加localhost：
echo 192.168.0.1:9200 | sed 's/^192.168.0.1/&localhost/'
输出：192.168.0.1localhost:9200

1. \w:

• \w 是一个元字符，表示 “word character”（单词字符）。
• 它匹配任意字母（包括大小写字母）、数字（0-9）和下划线（_）。
• 在大多数正则表达式实现中（如 Perl、Python、JavaScript 等），\w 等价于 [a-zA-Z0-9_]。

2. \ +:

• + 是一个量词，表示 “一个或多个” 的意思。
• 它指示前面的元素（在这个例子中是 \w）可以出现一次或多次。
• 在某些正则表达式的实现中，+ 是一个更常见的符号（例如，POSIX 和 Perl 正则表达式），而+ 通常在某些工具（如 sed 或 grep 的扩展模式）中使用。

2.5 子串匹配标记\1

匹配给定样式的其中一部分

#命中 digit 7，被替换成了 7。样式匹配到的子串是 7，(..) 用于匹配子串，
#对于匹配到的第一个子串就标记为 \1 ，依此类推匹配到的第二个结果就是 \2 
echo this is digit 7 in a number | sed 's/digit \([0-9]\)/\1/'
输出：this is 7 in a number# /1被匹配为多个小写字母组合，也就是aaa, 还有第二个匹配，也就是\2,最后两者显示位置互换
echo aaa BBB | sed 's/\([a-z]\+\) \([A-Z]\+\)/\2 \1/'
输出：BBB aaa# \1被匹配为replac 最后就是replacing ==> replacrssed -n 's/\(replac\)ing/\1rs/p' sample.txt 输出：6. For example, replacrs 'TEXT' with 'TEXT'.# 通过替换获取ipifconfig eth0 | sed -n '/inet /p' | sed 's/inet addr:\([0-9.]\+\).*/\1/'

2.6 大小写转换

\u：        首字母转换为大写
\U：  全部转换为大写
\l：         首字母转换为小写
\L：         全部转换为小写

sed 's/^[a-z]\+/\u&/' /etc/passwd #把/etc/passwd下匹配到的内容，首字母弄成大写sed 's/^[a-z]\+/\U&/' /etc/passwd #把/etc/passwd下匹配到的内容，全部弄成大写\l和\L跟上面是一样的

2.7 多点编辑：-e命令

e选项允许在同一行里执行多条命令：

sed -e 's/TEXT/test/g' -e '1d' sample.txt #TEXT替换text之后，再删除第一行内容
输出：
2. This is a sample test file for sed command practice.
3. It contains multiple lines of test.
4. Sed stands for Stream Editor.
5. You can use sed to replace test.
6. For example, replacing 'test' with 'test'.
7. This file includes some sample data.
8. Let's see how sed works with line numbers.
9. Line 9 is here just for testing.
10. This line has the word example.
11. Example lines can be very useful.
12. Sometimes, you may want to delete lines.
13. Deleting empty lines is a common task.
14. Regular expressions are powerful in sed.
15. You can also append or insert lines.
16. How about modifying the first line?
17. This is the 17th line of the test.
18. Line 18 contains special characters: !@#$
19. The end of this file is near.
20. Thank you for practicing with sed!

2.8 从文件读入：r命令

#命令的执行顺序
在''之外的文件中找出匹配test模式的行，如果找到了一行或者多行，那么将''里面的文件内容全部放在匹配行的后面
sed '/test/r sample.txt' sample.txt
输出：
1. Hello, World!
2. This is a sample TEXT file for sed command practice.
3. It contains multiple lines of TEXT.
4. Sed stands for Stream Editor.
5. You can use sed to replace TEXT.
6. For example, replacing 'TEXT' with 'TEXT'.
7. This file includes some sample data.
8. Let's see how sed works with line numbers.
9. Line 9 is here just for testing.
1. Hello, World!
2. This is a sample TEXT file for sed command practice.
3. It contains multiple lines of TEXT.
4. Sed stands for Stream Editor.
5. You can use sed to replace TEXT.
6. For example, replacing 'TEXT' with 'TEXT'.
7. This file includes some sample data.
8. Let's see how sed works with line numbers.
9. Line 9 is here just for testing.
10. This line has the word example.
11. Example lines can be very useful.
12. Sometimes, you may want to delete lines.
13. Deleting empty lines is a common task.
14. Regular expressions are powerful in sed.
15. You can also append or insert lines.
16. How about modifying the first line?
17. This is the 17th line of the TEXT.
18. Line 18 contains special characters: !@#$
19. The end of this file is near.
20. Thank you for practicing with sed!10. This line has the word example.
11. Example lines can be very useful.
12. Sometimes, you may want to delete lines.
13. Deleting empty lines is a common task.
14. Regular expressions are powerful in sed.
15. You can also append or insert lines.
16. How about modifying the first line?
17. This is the 17th line of the TEXT.
18. Line 18 contains special characters: !@#$
19. The end of this file is near.
20. Thank you for practicing with sed!

2.9 写入文件：w命令

# sample.txt中所有满足text的行都写入到sample1.txt中
sed '/test/w sample1.txt' sample.txt sample1.txt的文件内容：9. Line 9 is here just for testing.

2.10 追加（行下）：a\命令

sed '/^test/a\this is a test line' file在 test.conf 文件第2行之后插入 this is a test line：
sed -i '2a\this is a test line' test.conf

2.11 插入（行上）：i\命令

将 this is a test line 追加到以test开头的行前面：
sed '/^test/i\this is a test line' file
在test.conf文件第5行之前插入this is a test line：
sed -i '5i\this is a test line' test.conf