How to run a coroutine outside of an event loop?

There are two questions here: one is about awaiting a coroutine "at top-level", or more concretely in a development environment. The other is about running a coroutine without an event loop.

Regarding the first question, this is certainly possible in Python, just like it is possible in Chrome Canary Dev Tools - by the tool handling it via its own integration with the event loop. And indeed, IPython 7.0 and later support asyncio natively and you can use await coro() at top-level as expected.

Regarding the second question, it is easy to drive a single coroutine without an event loop, but it is not very useful. Let's examine why.

When a coroutine function is called, it returns a coroutine object. This object is started and resumed by calling its send() method. When the coroutine decides to suspend (because it awaits something that blocks), send() will return. When the coroutine decides to return (because it has reached the end or because it encountered an explicit return), it will raise a StopIteration exception with the value attribute set to the return value. With that in mind, a minimal driver for a single coroutine could look like this:

def drive(c):
    while True:
        try:
            c.send(None)
        except StopIteration as e:
            return e.value

This will work great for simple coroutines:

>>> async def pi():
...     return 3.14
... 
>>> drive(pi())
3.14

Or even for a bit more complex ones:

>>> async def plus(a, b):
...     return a + b
... 
>>> async def pi():
...     val = await plus(3, 0.14)
...     return val
... 
>>> drive(pi())
3.14

But something is still missing - none of the above coroutines ever suspend their execution. When a coroutine suspends, it allows other coroutines to run, which enables the event loop to (appear to) execute many coroutines at once. For example, asyncio has a sleep() coroutine that, when awaited, suspends the execution for the specified period:

async def wait(s):
    await asyncio.sleep(1)
    return s

>>> asyncio.run(wait("hello world"))
'hello world'      # printed after a 1-second pause

However, drive fails to execute this coroutine to completion:

>>> drive(wait("hello world"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in drive
  File "<stdin>", line 2, in wait
  File "/usr/lib/python3.7/asyncio/tasks.py", line 564, in sleep
    return await future
RuntimeError: await wasn't used with future

What happened is that sleep() communicates with the event loop by yielding a special "future" object. A coroutine awaiting on a future can only be resumed after the future has been set. The "real" event loop would do so by running other coroutines until the future is done.

To fix this, we can write our own sleep implementation that works with our mini event loop. To do this, we need to use an iterator to implement the awaitable:

class my_sleep:
    def __init__(self, d):
        self.d = d
    def __await__(self):
        yield 'sleep', self.d

We yield a tuple that will not be seen by the coroutine caller, but will tell drive (our event loop) what to do. drive and wait now look like this:

def drive(c):
    while True:
        try:
            susp_val = c.send(None)
            if susp_val is not None and susp_val[0] == 'sleep':
                time.sleep(susp_val[1])
        except StopIteration as e:
            return e.value

async def wait(s):
    await my_sleep(1)
    return s

With this version, wait works as expected:

>>> drive(wait("hello world"))
'hello world'

This is still not very useful because the only way to drive our coroutine is to call drive(), which again supports a single coroutine. So we might as well have written a synchronous function that simply calls time.sleep() and calls it a day. For our coroutines to support the use case of asynchronous programming, drive() would need to:

• support running and suspension of multiple coroutines
• implement spawning of new coroutines in the drive loop
• allow coroutines to register wakeups on IO-related events, such as a file descriptor becoming readable or writable - all the while supporting multiple such events without loss of performance

This is what the asyncio event loop brings to the table, along with many other features. Building an event loop from scratch is superbly demonstrated in this talk by David Beazley, where he implements a functional event loop in front of a live audience.

So after a bit of digging around, I think I found out the simplest solution to execute a coroutine globally.

If you >>> dir(coro) Python will print out the following attributes:

['__await__', '__class__', '__del__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__name__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'close', 'cr_await', 'cr_code', 'cr_frame', 'cr_origin', 'cr_running', 'send', 'throw']

A couple of attributes stand out, namely:

[
   '__await__',
   'close',
   'cr_await',
   'cr_code',
   'cr_frame',
   'cr_origin',
   'cr_running',
   'send',
   'throw'
]

After reading on what does yield (yield) do? and generally how generators work, I figured the send method must be the key.

So I tried to:

>>> the_actual_coro = coro()
<coroutine object coro at 0x7f5afaf55348> 

>>>the_actual_coro.send(None)

And it raised an interesting error:

Original exception was:
Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
StopIteration: a value

It actually gave me back the return value in an exception!

So I thought a very basic loop, well, it's more of a runner, can be implemented as such:

def run(coro):
    try:
        coro.send(None)
    except StopIteration as e:
        return e.value

Now, I can run a coroutine in a sync function, or even globally, not that I'd recommend doing that. But, it's interesting to know the simplest and lowest level you can go about running a coroutine

>>> run(coro())
'a value'

This however returns None when the coro has something to be awaited (which is really the very essence of being a coroutine).

I think that's probably because the event loop handles the awaitables of it's coroutines (coro.cr_frame.f_locals) by assigning them to futures and handling them seperately? which my simple run function obviously doesn't provide. I might be wrong in that regard. So please someone correct me if I'm wrong.