How to read specific part of large file in Python
This is my solution with variable width encoding. My CSV file contains a dictionary where each row is a new item.
def get_stuff(filename, count, start_index):
with open(filename, 'r') as infile:
reader = csv.reader(infile)
num = 0
for idx, row in enumerate(reader):
if idx >= start_index-1:
if num >= count:
return
else:
yield row
num += 1
You can seek
into the file the file and then read a certain amount from there. Seek allows you to get to a specific offset within a file, and then you can limit your read to only the number of bytes in that range.
with open(filename) as fin:
fin.seek(start_index)
data = fin.read(end_index - start_index)
That will only read that data that you're looking for.