Pandas, read CSV ignoring extra commas
You can use re.sub
to replace the first few commas with, say, the '|', save the intermediate results in a StringIO
then process that.
import pandas as pd
from io import StringIO
import re
for_pd = StringIO()
with open('MikeS159.csv') as mike:
for line in mike:
new_line = re.sub(r',', '|', line.rstrip(), count=7)
print (new_line, file=for_pd)
for_pd.seek(0)
df = pd.read_csv(for_pd, sep='|', header=None)
print (df)
I put the two lines from your question into a file to get this output.
0 1 2 3 4 5 6 \
0 061AE Active 1 2017_02_24 15_18_01 6 1 13
1 061AE Active 1 2017_02_24 15_18_01 6 1 13
7
0 some message
1 longer message, with commas
You can use the parameter usecols in the read_csv function to limit what columns you read in. For example:
import pandas as pd
pd.read_csv(path, usecols=range(8))
if you only want to read the first 8 columns.