With Clause for Multiprocessing in Python
with multiprocessing.Pool( ... ) as pool:
pool.starmap( ... )
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool
New in version 3.3: Pool objects now support the context management protocol – see Context Manager Types. enter() returns the pool object, and exit() calls terminate().
You can see an example at the bottom of the Pool
section.
Although its more than what the OP asked, if you want something that will work for both Python 2 and Python 3, you can use:
# For python 2/3 compatibility, define pool context manager
# to support the 'with' statement in Python 2
if sys.version_info[0] == 2:
from contextlib import contextmanager
@contextmanager
def multiprocessing_context(*args, **kwargs):
pool = multiprocessing.Pool(*args, **kwargs)
yield pool
pool.terminate()
else:
multiprocessing_context = multiprocessing.Pool
After that, you can use multiprocessing the regular Python 3 way, regardless of which version of Python you are using. For example:
def _function_to_run_for_each(x):
return x.lower()
with multiprocessing_context(processes=3) as pool:
results = pool.map(_function_to_run_for_each, ['Bob', 'Sue', 'Tim'])
print(results)
Now, this will work in Python 2 or Python 3.