When asyncio task gets stored after creation, exceptions from task get muted
This is because the exception only gets raised if the Task
is destroyed without ever having its result retrieved. When you assigned the Task
to a global variable, it will always have an active reference, and therefore never be destroyed. There's a docstring in asyncio/futures.py that goes into detail on this:
class _TracebackLogger:
"""Helper to log a traceback upon destruction if not cleared.
This solves a nasty problem with Futures and Tasks that have an
exception set: if nobody asks for the exception, the exception is
never logged. This violates the Zen of Python: 'Errors should
never pass silently. Unless explicitly silenced.'
However, we don't want to log the exception as soon as
set_exception() is called: if the calling code is written
properly, it will get the exception and handle it properly. But
we *do* want to log it if result() or exception() was never called
-- otherwise developers waste a lot of time wondering why their
buggy code fails silently.
An earlier attempt added a __del__() method to the Future class
itself, but this backfired because the presence of __del__()
prevents garbage collection from breaking cycles. A way out of
this catch-22 is to avoid having a __del__() method on the Future
class itself, but instead to have a reference to a helper object
with a __del__() method that logs the traceback, where we ensure
that the helper object doesn't participate in cycles, and only the
Future has a reference to it.
The helper object is added when set_exception() is called. When
the Future is collected, and the helper is present, the helper
object is also collected, and its __del__() method will log the
traceback. When the Future's result() or exception() method is
called (and a helper object is present), it removes the the helper
object, after calling its clear() method to prevent it from
logging.
If you want to see/handle the exception, just use add_done_callback
to handle the result of the task, and do whatever is necessary when you get an exception:
import asyncio
def handle_result(fut):
if fut.exception():
fut.result() # This will raise the exception.
def schedule_something():
global f
tsk = asyncio.async(do_something())
tsk.add_done_callback(handle_result)
f = tsk
@asyncio.coroutine
def do_something():
raise Exception()
loop = asyncio.get_event_loop()
loop.call_soon(schedule_something)
loop.run_forever()
loop.close()
Thanks, @dano. Here's a drop-in replacement for asyncio.create_task
that does this automatically -
def create_task(coro):
task = asyncio.create_task(coro)
return TaskWrapper(task)
class TaskWrapper:
def __init__(self, task):
self.task = task
task.add_done_callback(self.on_task_done)
def __getattr__(self, name):
return getattr(self.task, name)
def __await__(self):
self.task.remove_done_callback(self.on_task_done)
return self.task.__await__()
def on_task_done(self, fut: asyncio.Future):
if fut.cancelled() or not fut.done():
return
fut.result()
def __str__(self):
return f"TaskWrapper<task={self.task}>"
Updated version of the given example -
async def do_something():
raise Exception()
async def schedule_something():
global f
tsk = create_task(do_something())
f = tsk # If this line is commented out, exceptions can be heard.
asyncio.run(schedule_something())
$ python test.py
Exception in callback TaskWrapper.on_task_done(<Task finishe...n=Exception()>)
handle: <Handle TaskWrapper.on_task_done(<Task finishe...n=Exception()>)>
Traceback (most recent call last):
File "/Users/dev/.pyenv/versions/3.8.1/lib/python3.8/asyncio/events.py", line 81, in _run
self._context.run(self._callback, *self._args)
File "/Users/dev/Projects/dara/server/bot/async_util.py", line 21, in on_task_done
fut.result()
File "/Users/dev/Projects/dara/server/test.py", line 7, in do_something
raise Exception()
Exception