Prevent TextIOWrapper from closing on GC in a Py2/Py3 compatible way
Just detach your TextIOWrapper()
object before letting it be garbage collected:
def mangle(x):
wrapper = io.TextIOWrapper(x)
wrapper.detach()
The TextIOWrapper()
object only closes streams it is attached to. If you can't alter the code where the object goes out of scope, then simply keep a reference to the TextIOWrapper()
object locally and detach at that point.
If you must subclass TextIOWrapper()
, then just call detach()
in the __del__
hook:
class DetachingTextIOWrapper(io.TextIOWrapper):
def __del__(self):
self.detach()
EDIT:
Just call detach
first, thanks martijn-pieters!
It turns out there is basically nothing that can be done about the deconstructor calling close
in Python 2.7. This is hardcoded into the C code. Instead we can modify close
such that it won't close the buffer when __del__
is happening (__del__
will be executed before _PyIOBase_finalize
in the C code giving us a chance to change the behaviour of close
). This lets close
work as expected without letting the GC close the buffer.
class SaneTextIOWrapper(io.TextIOWrapper):
def __init__(self, *args, **kwargs):
self._should_close_buffer = True
super(SaneTextIOWrapper, self).__init__(*args, **kwargs)
def __del__(self):
# Accept the inevitability of the buffer being closed by the destructor
# because of this line in Python 2.7:
# https://github.com/python/cpython/blob/2.7/Modules/_io/iobase.c#L221
self._should_close_buffer = False
self.close() # Actually close for Python 3 because it is an override.
# We can't call super because Python 2 doesn't actually
# have a `__del__` method for IOBase (hence this
# workaround). Close is idempotent so it won't matter
# that Python 2 will end up calling this twice
def close(self):
# We can't stop Python 2.7 from calling close in the deconstructor
# so instead we can prevent the buffer from being closed with a flag.
# Based on:
# https://github.com/python/cpython/blob/2.7/Lib/_pyio.py#L1586
# https://github.com/python/cpython/blob/3.4/Lib/_pyio.py#L1615
if self.buffer is not None and not self.closed:
try:
self.flush()
finally:
if self._should_close_buffer:
self.buffer.close()
My previous solution here used _pyio.TextIOWrapper
which is slower than the above because it is written in Python, not C.
It involved simply overriding __del__
with a noop which will also work in Py2/3.