How to use an async for loop to iterate over a list?
Please note, that Vincents answer has a partial problem:
You must have a splatter operator infront of the map
function, otherwise asyncio.gather
would try to use the list as whole. So do it like this:
results = await asyncio.gather(*map(fetch, url))
Use asyncio.as_completed:
for future in asyncio.as_completed(map(fetch, urls)):
result = await future
Or asyncio.gather:
results = await asyncio.gather(map(fetch, urls))
EDIT: If you don't mind having an external dependency, you can use aiostream.stream.map:
from aiostream import stream, pipe
async def fetch_many(urls):
xs = stream.iterate(urls) | pipe.map(fetch, ordered=True, task_limit=10)
async for result in xs:
print(result)
You can control the amount of fetch
coroutine running concurrently using the task_limit
argument, and choose whether to get the results in order, or as soon as possible.
See more examples in this demonstration and the documentation.
Disclaimer: I am the project maintainer.