shell第一列相同即判断为重复，只取其中一条数据
例如 123456.txt内容如下
90115601,2137786735 36684
90115601,2137786735 36684
90118301,17751822 34898
90145133,2050186877 3847
90145133,2050186877 3847
90151825,3580430507 37974
90151825,3580430507 37974
90188754,2987861307 24533
90188754,2987861307 24533
910910,1303405168 1855
910910,514876016 1555
991314,1879979639 49504
991314,4119267805 50922
998008,1453233015 13818
取第一条数据
tac 123456.txt |awk -F, '!a[$1]++' |sort -t"," -k1
取最后一条数据
cat 123456.txt |awk -F, '!a[$1]++' |sort -t"," -k1
转载于:https://blog.51cto.com/6226001001/1904482