Kotlin Coroutines : Waiting for multiple threads to finish

You don't need to manually keep track of your cuncurrent jobs if you use the concept of structured concurrency. Assuming that your processPages function performs some kind of blocking IO, you can encapsulate your code into the following suspending function, which executes your code in an IO dispatcher designed for this kind of work:

suspend fun processAllPages() = withContext(Dispatchers.IO) { 
    // withContext waits for all children coroutines 
    launch { processPages(urls, collection) }
    launch { processPages(urls, collection2) }
    launch { processPages(urls, collection3) }
}

Now, from if a topmost function of your application is not already a suspending function, then you can use runBlocking to call processAllPages:

runBlocking {
    processAllPages()
}

You can use async builder function to process a load of data in parallel:

class Presenter {
    private var job: Job = Job()
    private var scope = CoroutineScope(Dispatchers.Main + job) // creating the scope to run the coroutine. It consists of Dispatchers.Main (coroutine will run in the Main context) and job to handle the cancellation of the coroutine.

    fun runInParallel() {
        scope.launch { // launch a coroutine
            // runs in parallel
            val deferredList = listOf(
                    scope.asyncIO { processPages(urls, collection) },
                    scope.asyncIO { processPages(urls, collection2) },
                    scope.asyncIO { processPages(urls, collection3) }
            )

            deferredList.awaitAll() // wait for all data to be processed without blocking the UI thread

            // do some stuff after data has been processed, for example update UI
        }
    }

    private fun processPages(...) {...}

    fun cancel() {
        job.cancel() // invoke it to cancel the job when you don't need it to execute. For example when UI changed and you don't need to process data
    }
}

Extension function asyncIO:

fun <T> CoroutineScope.asyncIO(ioFun: () -> T) = async(Dispatchers.IO) { ioFun() } // CoroutineDispatcher - runs and schedules coroutines

GlobalScope.launch is not recommended to use unless you want the coroutine to be operating on the whole application lifetime and not cancelled prematurely.

Edit: as mentioned by Roman Elizarov you can try not to use awaitAll() function unless you want to update UI or do something else right away after all data are processed.


Following approach can be used.

fun myTask() {
    GlobalScope.launch {
        val task = listOf(
            async {

            },
            async {

            }
        )
        task.awaitAll()

    }
}