What is Clojure volatile?

The new volatile is as close as a real "variable" (as it is from many other programming languages) as it gets for clojure.

From the announcement:

there are a new set of functions (volatile!, vswap!, vreset!, volatile?) to create and use volatile "boxes" to hold state in stateful transducers. Volatiles are faster than atoms but give up atomicity guarantees so should only be used with thread isolation.

For instance, you can set/get and update them just like you would do with a variable in C. The only addition (and hence the name) is the volatile keyword to the actual java object.

This is to prevent the JVM from optimization and makes sure that it reads the memory location every time it is accessed. From the JIRA ticket:

Clojure needs a faster variant of Atom for managing state inside transducers. That is, Atoms do the job, but they provide a little too much capability for the purposes of transducers. Specifically the compare and swap semantics of Atoms add too much overhead. Therefore, it was determined that a simple volatile ref type would work to ensure basic propagation of its value to other threads and reads of the latest write from any other thread. While updates are subject to race conditions, access is controlled by JVM guarantees.

Solution overview: Create a concrete type in Java, akin to clojure.lang.Box, but volatile inside supports IDeref, but not watches etc.

This mean, a volatile! can still be accessed by multiple threads (which is necessary for transducers) but it does not allow to be changed by these threads at the same time since it gives you no atomic updates.

The semantics of what volatile does is very well explained in a java answer:

there are two aspects to thread safety: (1) execution control, and (2) memory visibility. The first has to do with controlling when code executes (including the order in which instructions are executed) and whether it can execute concurrently, and the second to do with when the effects in memory of what has been done are visible to other threads. Because each CPU has several levels of cache between it and main memory, threads running on different CPUs or cores can see "memory" differently at any given moment in time because threads are permitted to obtain and work on private copies of main memory.

Now let's see why not use var-set or transients:

Volatile vs var-set

Rich Hickey didn't want to give truly mutable variables:

Without mutable locals, people are forced to use recur, a functional looping construct. While this may seem strange at first, it is just as succinct as loops with mutation, and the resulting patterns can be reused elsewhere in Clojure, i.e. recur, reduce, alter, commute etc are all (logically) very similar. [...] In any case, Vars are available for use when appropriate.

And thus creating with-local-vars, var-set etc.. The problem with these is that they're true vars and the doc string of var-set tells you:

The var must be thread-locally bound.

This is, of course, not an option for core.async which potentially executes on different threads. They're also much slower because they do all those checks.

Why not use transients

Transients are similar in that they don't allow concurrent access and optimize mutating a data structure. The problem is that transient only work with collection that implement IEditableCollection. That is they're simply to avoid expensive intermediate representation of the collection data structures. Also remember that transients are not bashed into place and you still need some memory location to store the actual transient. Volatiles are often used to simply hold a flag or the value of the last element (see partition-by for instance)

Summary:

Volatile's are nothing else but a wrapper around java's volatile and have thus the exact same semantics. Don't ever share them. Use them only very carefully.

Volatiles are a "faster atom" with no atomicity guarantees. They were introduced as atoms were considered too slow to hold state in transducers.

there are a new set of functions (volatile!, vswap!, vreset!, volatile?) to create and use volatile "boxes" to hold state in stateful transducers. Volatiles are faster than atoms but give up atomicity guarantees so should only be used with thread isolation