How to read a specific line number in a csv with pandas
Looking in the pandas documentation, there is a parameter for read_csv function:
skiprows
If a list is assigned to this parameter it will skip the line indexed by the list:
skiprows = [0,1]
This will skip the first one and the second line.
Thus a combination of nrow
and skiprows
allow to read each line in the dataset separately.
You are using nrows = 1
, wich means "Number of rows of file to read. Useful for reading pieces of large files"
So you are telling it to read only the first row and stop.
You should just remove the argument to read all the csv file into a DataFrame and then go line by line.
See the documentation for more details on usage : https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html
One way could be to read part by part of your file and store each part, for example:
df1 = pd.read_csv("mydata.csv", nrows=10000)
Here you will skip the first 10000 rows that you already read and stored in df1, and store the next 10000 rows in df2.
df2 = pd.read_csv("mydata.csv", skiprows=10000 nrows=10000)
dfn = pd.read_csv("mydata.csv", skiprows=(n-1)*10000, nrows=10000)
Maybe there is a way to introduce this idea into a for or while loop.