Python multiprocessing: How to close the multiprocessing pool on exception

You have to use: del Pool after Pool.terminate & Pool.join

This will solve your problem

I got the solution for this - Catch exception at the parent process.

try:
   pool = Pool(processes=4)
   from functools import partial
   param_data = "Test Value"
   func = partial(test_function, param_data)
   r = pool.map(func, range(3))
except Exception as e:
   pool.close()
pool.close()

And also add a try/catch in the child process function:

def test_function(param_data,index):
    try:
       # The process here;
    except Exception as e:
       raise Exception(e.message)          

The except block here looks redundant but it is not. The reason is, some of the exceptions raised by child process were not thrown to the parent process.

For example, I was raising a custom exception in the function and it was not thrown to the parent process, So it is advised to catch all exceptions within the child process and raise the standard Exception from there, Then handle it at parent process, where you can close the process pool or do other cleanups.