What's the correct way to clean up after an interrupted event loop?
When you CTRL+C, the event loop gets stopped, so your calls to t.cancel()
don't actually take effect. For the tasks to be cancelled, you need to start the loop back up again.
Here's how you can handle it:
import asyncio
@asyncio.coroutine
def shleepy_time(seconds):
print("Shleeping for {s} seconds...".format(s=seconds))
yield from asyncio.sleep(seconds)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
# Side note: Apparently, async() will be deprecated in 3.4.4.
# See: https://docs.python.org/3.4/library/asyncio-task.html#asyncio.async
tasks = asyncio.gather(
asyncio.async(shleepy_time(seconds=5)),
asyncio.async(shleepy_time(seconds=10))
)
try:
loop.run_until_complete(tasks)
except KeyboardInterrupt as e:
print("Caught keyboard interrupt. Canceling tasks...")
tasks.cancel()
loop.run_forever()
tasks.exception()
finally:
loop.close()
Once we catch KeyboardInterrupt
, we call tasks.cancel()
and then start the loop
up again. run_forever
will actually exit as soon as tasks
gets cancelled (note that cancelling the Future
returned by asyncio.gather
also cancels all the Futures
inside of it), because the interrupted loop.run_until_complete
call added a done_callback
to tasks
that stops the loop. So, when we cancel tasks
, that callback fires, and the loop stops. At that point we call tasks.exception
, just to avoid getting a warning about not fetching the exception from the _GatheringFuture
.
Note for Python 3.7+: The below is now implemented as part of the standard library asyncio.run
function – Replace the below with sys.exit(loop.run(amain(loop)))
once you are ready to upgrade! (If you want to print the message, simply move that try…except
-clause into amain
.)
Updated for Python 3.6+: Add call to loop.shutdown_asyncgens
to avoid memory leaks by asynchronous generators that weren't fully used.
The following solution, inspired by some of the other answers, should work in almost all cases and does not depend on you manually keeping track of tasks that need to be cleaned up on Ctrl+C:
loop = asyncio.get_event_loop()
try:
# Here `amain(loop)` is the core coroutine that may spawn any
# number of tasks
sys.exit(loop.run_until_complete(amain(loop)))
except KeyboardInterrupt:
# Optionally show a message if the shutdown may take a while
print("Attempting graceful shutdown, press Ctrl+C again to exit…", flush=True)
# Do not show `asyncio.CancelledError` exceptions during shutdown
# (a lot of these may be generated, skip this if you prefer to see them)
def shutdown_exception_handler(loop, context):
if "exception" not in context \
or not isinstance(context["exception"], asyncio.CancelledError):
loop.default_exception_handler(context)
loop.set_exception_handler(shutdown_exception_handler)
# Handle shutdown gracefully by waiting for all tasks to be cancelled
tasks = asyncio.gather(*asyncio.Task.all_tasks(loop=loop), loop=loop, return_exceptions=True)
tasks.add_done_callback(lambda t: loop.stop())
tasks.cancel()
# Keep the event loop running until it is either destroyed or all
# tasks have really terminated
while not tasks.done() and not loop.is_closed():
loop.run_forever()
finally:
loop.run_until_complete(loop.shutdown_asyncgens())
loop.close()
The above code will obtain all currently tasks from the event loop using asyncio.Task.all_tasks
and place them in a single combined future using asyncio.gather
. All tasks in that future (which are all currently running tasks) are then canceled using the future's .cancel()
method. The return_exceptions=True
then ensures that all the received asyncio.CancelledError
exceptions are stored instead of causing the future to become errored.
The above code will also override the default exception handler to prevent the generated asyncio.CancelledError
exceptions from being logged.
Update from 2020-12-17: Dropped compatiblity code for Python 3.5.
In Python 3.7+ it's recommended that you use asyncio.run
to start an async main function.
asyncio.run
will take care of creating the event loop for your program and ensure that the event loop is closed and all tasks are cleaned when the main function exits (including due to a KeyboardInterrupt
exception).
It is roughly analogous to the following (see asyncio/runners.py
):
def run(coro, *, debug=False):
"""`asyncio.run` is new in Python 3.7"""
loop = asyncio.get_event_loop()
try:
loop.set_debug(debug)
return loop.run_until_complete(coro)
finally:
try:
all_tasks = asyncio.gather(*asyncio.all_tasks(loop), return_exceptions=True)
all_tasks.cancel()
with contextlib.suppress(asyncio.CancelledError):
loop.run_until_complete(all_tasks)
loop.run_until_complete(loop.shutdown_asyncgens())
finally:
loop.close()