How to wait for a ThreadPoolExecutor to finish

Your issue seems to be that you are not calling shutdown after you have submitted all of the jobs to your pool. Without shutdown() your awaitTermination will always return false.

ThreadPoolExecutor ex =
    new ThreadPoolExecutor(limit, limit, 20, TimeUnit.SECONDS, q);
for (int i = 0; i < limit; i++) {
  ex.execute(new RunnableObject(i + 1));
}
// you are missing this line!!
ex.shutdown();
ex.awaitTermination(2, TimeUnit.SECONDS);

You can also do something like the following to wait for all your jobs to finish:

List<Future<Object>> futures = new ArrayList<Future<Object>>();
for (int i = 0; i < limit; i++) {
  futures.add(ex.submit(new RunnableObject(i + 1), (Object)null));
}
for (Future<Object> future : futures) {
   // this joins with the submitted job
   future.get();
}
...
// still need to shutdown at the end
ex.shutdown();

Also, because you are sleeping for 2354 milliseconds but only waiting for the termination of all of the jobs for 2 SECONDS, awaitTermination will always return false. I would use Long.MAX_VALUE to wait for the jobs to finish.

Lastly, it sounds like you are worrying about created a new ThreadPoolExecutor and you instead want to reuse the first one. Don't be. The GC overhead is going to be extremely minimal compared to any code that you write to detect if the jobs are finished.

---

To quote from the javadocs, ThreadPoolExecutor.shutdown():

Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted. Invocation has no additional effect if already shut down.

In the ThreadPoolExecutor.awaitTermination(...) method, it is waiting for the state of the executor to go to TERMINATED. But first the state must go to SHUTDOWN if shutdown() is called or STOP if shutdownNow() is called.

It's nothing to do with the executor itself. Just use the interface's java.util.concurrent.ExecutorService.invokeAll(Collection<? extends Callable<T>>). It will block until all the Callables are finished.

Executors are meant to be long-lived; beyond the lifetime of a group of tasks. shutdown is for when the application is finished and cleaning up.

You should loop on awaitTermination

ExecutorService threads;
// ...
// Tell threads to finish off.
threads.shutdown();
// Wait for everything to finish.
while (!threads.awaitTermination(10, TimeUnit.SECONDS)) {
  log.info("Awaiting completion of threads.");
}

Here's a variant on the accepted answer that handles retries if/when InterruptedException is thrown:

executor.shutdown();

boolean isWait = true;

while (isWait)
{
    try
    {             
        isWait = !executor.awaitTermination(10, TimeUnit.SECONDS);
        if (isWait)
        {
            log.info("Awaiting completion of bulk callback threads.");
        }
    } catch (InterruptedException e) {
        log.debug("Interruped while awaiting completion of callback threads - trying again...");
    }
}