Python Multiprocessing - Why are my processes are not returning/finishing?
You are calling join()
on all the processes before you're get()
ing the results. When a Queue's buffer fills up, it can block when the data is flushed to the underlying pipe. If you join()
a process blocked that way from your consumer process you have a deadlock because the process can only exit after all data has been written.
Move the call to join to the end of your main()
then it should work:
def main():
output = mp.Queue()
processes = [mp.Process(target=longRunningTask, args=(x,output,)) for x in range(4)]
for p in processes:
p.start()
results = [output.get() for p in processes]
print("This never shows up")
for p in processes:
p.join()