Lumenworks Csv reader To read columns with same Names Or avoid ` An item with the same key has already been added`
You've stumped me -- I don't know of any CSV parser that accounts for duplicate column headers, and I've tested quite a few of them. There are CSV parsers which will give you raw column data, though, and with some leg work you can use this as a building block to get your data into a friendlier state.
This will return a sequence of Dictionary<string, List<string>>
, one for each record, with the key being the header and the list being all the columns with the same header:
using System.IO;
using System.Collections.Generic;
using Ctl.Data;
static IEnumerable<Dictionary<string, List<string>>> ReadCsv(string filePath)
{
using (StreamReader sr = new StreamReader(filePath))
{
CsvReader csv = new CsvReader(sr);
// first read in the header.
if (!csv.Read())
{
yield break; // an empty file, break out early.
}
RowValue header = csv.CurrentRow;
// now the records.
while (csv.Read())
{
Dictionary<string, List<string>> dict =
new Dictionary<string, List<string>>(header.Count);
RowValue record = csv.CurrentRow;
// map each column to a header value
for (int i = 0; i < record.Count; ++i)
{
// if there are more values in the record than the header,
// assume an empty string as header.
string headerValue = (i < header.Count ? header[i].Value : null)
?? string.Empty;
// get the list, or create if it doesn't exist.
List<string> list;
if (!dict.TryGetValue(headerValue, out list))
{
dict[headerValue] = list = new List<string>();
}
// finally add column value to the list.
list.Add(record[i].Value);
}
yield return dict;
}
}
}
I'm not familiar enough with Lumenworks -- this uses Ctl.Data, which I know will allow for formatted JSON data and any other weirdness within columns so long as it is properly quoted. (disclaimer: I'm the author of Ctl.Data)