Can I use map / imap / imap_unordered with functions with no arguments?
map
function's first argument should be a function and it should accept one argument. It is mandatory because, the iterable passed as the second argument will be iterated and the values will be passed to the function one by one in each iteration.
So, your best bet is to redefine f
to accept one argument and ignore it, or write a wrapper function with one argument, ignore the argument and return the return value of f
, like this
from multiprocessing import Pool
def f(): # no argument
return 1
def throw_away_function(_):
return f()
print(Pool(2).map(throw_away_function, range(10)))
# [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
You cannot use lamdba
functions with pools because they are not picklable.
You can use pool.starmap()
instead of .map()
like so:
from multiprocessing import Pool
def f(): # no argument
return 1
print Pool(2).starmap(f, [() for _ in range(10)])
starmap
will pass all elements of the given iterables as arguments to f
. The iterables should be empty in your case.
Is there anything wrong with using Pool.apply_async
?
with multiprocessing.Pool() as pool:
future_results = [pool.apply_async(f) for i in range(n)]
results = [f.get() for f in future_results]