Meaning of "OSError: telling position disabled by next() call" error?
The message means exactly what it says: because you have called next()
on the file, the use of tell()
on that file has been disabled.
It might not look like you've called next
, but the for
loop calls it implicitly. A for
loop:
for element in thing:
do_stuff_with(element)
is syntactical sugar for
iterator = iter(thing) # the real implementation doesn't use a variable
while True:
try:
element = next(iterator) # here's the next() call
except StopIteration:
break
do_stuff_with(element)
For a file, iter(file)
returns the file, and the loop calls next
on the file.
As for why calling next
disables tell()
, this is for efficiency. It only happens for text files (specifically io.TextIOWrapper
), which have to do a bunch of extra work to support tell
; turning off tell
support lets them skip that work. The original commit message for the change that made next
disable tell
is "Speed up next() by disabling snapshot updating then.", indicating it's for efficiency.
For historical context, previous Python versions used a hidden buffer for next
that tell
and other file methods didn't account for, causing tell
(and other file methods) to produce not-very-meaningful results during iteration over a file. The current IO implementation would be able to support tell()
during iteration, but io.TextIOWrapper
prevents such calls anyway. The historical incompatibility between next
and other methods likely contributed to why it was considered reasonable to disable parts of file functionality during iteration.
You didn't ask for workarounds, but for the benefit of people who end up on this page looking for a workaround, I'll mention that
for line in iter(file.readline, ''):
...
will let you iterate over the lines of a text file without disabling tell
. (You can use for line in iter(file.readline, b'')
for binary files, but there's not much point, because the tell
disabling mechanism isn't there for binary files.)
If your text file is too large, there are two solutions according to this answer:
- Using
file.readline()
instead ofnext()
with open(path, mode) as file:
while True:
line = file.readline()
if not line:
break
file.tell()
- Using
offset += len(line)
instead offile.tell()
offset = 0
with open(path, mode) as file:
for line in file:
offset += len(line)