Is there a robust command line tool for processing csv files?
I am probably a little bit too late, but there is another tool worth mentioning: csvkit.
It has a lot of command line tools that can:
- reformatting CSV files,
- convert to and from CSV from various formats (JSON, SQL, XLS),
- the equivalent of
cut
,grep
,sort
and others, but CSV-aware, - join different CSV files,
- do general SQL queries on data from CSV files.
You can use Python's csv
module.
A simple example:
import csv
reader = csv.reader(open("test.csv", "r"))
for row in reader:
for col in row:
print col
Sounds like a job for Perl with Text::CSV
.
perl -MText::CSV -pe '
BEGIN {$csv = Text::CSV->new();}
$csv->parse($_) or die;
@fields = $csv->fields();
print @fields[1,3];
'
See the documentation for how to handle column names. The separator and quoting style can be tuned with parameters to new
. See also Text::CSV::Separator
for separator guessing.