Filter columns in string with awk piped with xargs
In your expression
"cat my.log | grep @ | tail -n 1 | awk '{ print $3 }'"
...the double-quotes around this string mean that the single-quotes are treated as literals. They don't protect $3
from the shell, so it's being expanded as an environment variable. As $3
is not actually defined by the shell (unless this is in a script that you've invoked with 3 arguments), it becomes the empty string, and the awk
expression is simply { print }
, printing the whole line.
You could fix this by escaping the $
:
ls *.csv | xargs -I@ bash -c "cat my.log | grep @|tail -n 1|awk '{print \$3}'"
...or by moving the awk
out of the xargs
expression:
ls *.csv | xargs -I@ bash -c "cat my.log | grep @|tail -n 1"|awk '{print $3}'
Piping the output of ls
into xargs
is a bad idea (in fact, doing anything with the output of ls
other than simply viewing it in your terminal is a bad idea). If you absolutely must do something like this, at least use something like find . -maxdepth 1 -type f -iname '*.csv' -print0
and pipe that into xargs -0r
.
But, in thise case, you don't need to do it at all because the filenames of your .csv files are already in my.log
.
In awk:
#!/usr/bin/awk -f
{ seen[$1] = $3 }
END {
for (f in seen) { print seen[f] };
}
or as a one-liner:
$ awk '{seen[$1] = $3}; END {for (f in seen) { print seen[f] };}' my.log
c
d
d
These will print the last value seen in column 3 for each file listed in column 1.
If you want it to print only the first value seen in column 3, change it to:
!seen[$1] { seen[$1] = $3 }
If you don't want to use find | xargs
and
you really need to use the filenames of all the .csv
files currently in the current directory, one alternative is to do something like this:
#!/usr/bin/perl
use strict;
my $logfile=shift; # get the first arg (the logfile name)
my $re=join("|",@ARGV); # turn the remaining args into a regular expression
@ARGV=$logfile; # set the logfile name as the sole cmd-line argument.
my %seen=();
while(<>) {
next unless (m/^($re)/o); # ignore any filenames that weren't on the cmd line.
my(@F) = split;
$seen{$F[0]} = $F[2]; # perl arrays start from 0, not 1.
};
foreach my $file (sort keys %seen) {
print $seen{$file}, "\n";
};
save it as, e.g. nandro.pl
, make it executable with chmod +x
and run it as:
$ ./nandro.pl my.log *.csv
c
d
d