When to use core.async in Clojure?

It's important to point out that there are 2 common meanings associated to the word 'async' in programming circles:

1. asynchronous messaging: Systems in which components send messages without expecting a response from their consumers, and often without even knowing who the consumers are (via queues)
2. non-blocking (a.k.a event-driven) I/O: Programs structured in such a way that they don't block expensive computational resources (threads, cores, ...) while awaiting a response. There are several approches, with varying levels of abstraction, for dealing with such systems: callback-based APIs (low-level, difficult to manage because based on side-effects), Promise/Futures/Deferreds (representations of values that we don't yet have, more manageable as they are value-based), Green Threads ('logical' threads which emulate ordinary control flow, but are inexpensive)

core.async is very opinionated towards 1 (asynchronous messaging via queues), and provides a macro for implementing Green Threading (the go macro).

From my experience, if non-blocking is all you need, I would personally recommend starting with Manifold, which makes fewer assumptions about your use case, then use core.async for the more advanced use cases where it falls short; note that both libraries interoperate well.

I find it useful for fine-grained, configurable control of side-effect parallelism on the JVM.

e.g. If the following executes a read from Cassandra, returning an async/chan:

(arche.async/execute connection :key {:values {:id "1"}})

Then the following performs a series of executes in parallel, where that parallelism is configurable, and the results are returned in-order.

(async/pipeline-async
    n-parallelism
    out
    #(arche/execute connection :key {:values %1 :channel %2})
    (async/to-chan [{:id "1"} {:id "2"} {:id "3"} ... ]))

Probably quite particular to my niche, but you get the idea.

• https://github.com/troy-west/arche

You may wish to read this:

• Clojure core.async Channels Introductory blog by Rich Hickey
• Mastering Concurrent Processes with core.async Brave Clojure entry

The best use case for core.async is ClojureScript, since it allows you to simulate multi-threaded programming and avoid Callback Hell.

In JVM Clojure, core.async can also by handy where you want a (lightweight) producer-consumer architecure. Of course, you could always use native Java queues for that, as well.

Core.async provides building blocks for socket like programming which is useful for coordinating producer/consumer interaction in Clojure. Core.async's lightweight threads (go blocks) let you write imperative style code reading from channels instead of using callbacks in the browser. On the JVM, the lightweight threads let you utilize your full CPU threads.

You can see an example of a full-stack CLJ/CLJS producer/consumer Core.async chat room here: https://github.com/briangorman/hablamos

Another killer feature of core.async is the pipeline feature. Often in data processing you will have the initial processing stage take up most of the CPU time, while later stages e.g. reducing will take up significantly less. With the async pipeline feature, you can split up the processing over channels to add parallelization to your pipeline. Core.async channels work with transducers, so channels play nice with the rest of the language now.