Remove duplicates values within a field
A sed
one:
sed '
s/[^[:blank:]]\{1,\}/,&,/g;:1
s/\(\(,[^,[:blank:]]*\)\(,[^,[:blank:]]*\)*\)\2,/\1,/;t1
s/,\([^[:blank:]]*\),/\1/g'
(it processes all the fields that contain ,
characters and preserves spacing)
With perl
:
perl -MList::MoreUtils=uniq -pe 's{\S*,\S*}{join ",", uniq split ",", $&}ge'
(it processes all the fields that contain ,
characters and preserves spacing)
Another perl
solution:
perl -anle '
print "$F[0] ", join ",", grep {!$seen{$_}++} split ",",$F[1];
%seen=();
' file
A 1,2,3,45,8
B 5,6,2,3,7