Show the progress of a Python multiprocessing pool imap_unordered call?
There is no need to access private attributes of the result set:
from __future__ import division
import sys
for i, _ in enumerate(p.imap_unordered(do_work, xrange(num_tasks)), 1):
sys.stderr.write('\rdone {0:%}'.format(i/num_tasks))
My personal favorite -- gives you a nice little progress bar and completion ETA while things run and commit in parallel.
from multiprocessing import Pool
import tqdm
pool = Pool(processes=8)
for _ in tqdm.tqdm(pool.imap_unordered(do_work, tasks), total=len(tasks)):
pass