uniq

uniq是删除重复内容

 

 

1、查看uniq的帮助

-bash-3.2# uniq --help
用法：uniq [选项]... [输入 [输出]]
Discard all but one of successive identical lines from INPUT (or
standard input), writing to OUTPUT (or standard output).

长选项必须用的参数在使用短选项时也是必须的。
  -c, --count           prefix lines by the number of occurrences
  -d, --repeated        only print duplicate lines
  -D, --all-repeated[=delimit-method] print all duplicate lines
                        delimit-method={none(default),prepend,separate}
                        Delimiting is done with blank lines.
  -f, --skip-fields=N   avoid comparing the first N fields
  -i, --ignore-case     ignore differences in case when comparing
  -s, --skip-chars=N    avoid comparing the first N characters
  -u, --unique          only print unique lines
  -w, --check-chars=N   compare no more than N characters in lines
      --help     显示此帮助信息并退出
      --version  输出版本信息并退出

 

 

2、只显示唯一的行

-bash-3.2# cat 1.txt
1:a:tail
2:b:one
3:c:less
4:d:more
4:d:more
4:d:cat
-bash-3.2# uniq -u 1.txt
1:a:tail
2:b:one
3:c:less
4:d:cat

 

 

3、统计次数

-bash-3.2# cat 1.txt |uniq -c
      1 1:a:tail
      1 2:b:one
      1 3:c:less
      2 4:d:more
      1 4:d:cat

 

 

4、查找出重复的行

-bash-3.2# cat 1.txt |uniq -d
4:d:more

 

 

5、只显示第一行

-bash-3.2# cat 1.txt
1:a:tail
2:b:one
3:c:less
4:d:more
4:d:more
4:d:cat
-bash-3.2# uniq -f 1 1.txt
1:a:tail

 

 

6、指定跳过前N个字符和指定比较的最大字符

-bash-3.2# cat 1.txt
1:a:tail
2:b:one
3:c:less
4:d:more
4:d:more
4:d:cat
-bash-3.2# sort 1.txt |uniq -s 1 -w 2
1:a:tail
2:b:one
3:c:less
4:d:cat