What makes random variables exchangeable and what is implied by exchangeability?

Exchangeability implies being identically distributed, but not the reverse. An exchangeable sequence has extra symmetry.

For instance, below I've written the joint mass function of $(X,Y)$, where $X$ and $Y$ take values in $\{0,1,2\}$. Checking the marginals shows that they are identically distributed, but $$\mathbb{P}(X=0,Y=2)=1/7\neq \mathbb{P}(Y=0,X=2)=0/7.$$ That is, the random variables $X$ and $Y$ have the same distribution, but the random vectors $(X,Y)$ and $(Y,X)$ don't.

$$\begin{array}{c|ccc} x\backslash y&0&1&2\cr \hline 0&0/7&1/7&1/7\\ 1&2/7&0/7&1/7\\ 2&0/7&2/7&0/7\\ \end{array} $$

See here and here for further applications of exchangeability.

A simple and rather general way to build an exchangeable random vector $X=(X_k)_{1\leqslant k\leqslant n}$ is to start from an i.i.d. sequence $(Y_k)_{1\leqslant k\leqslant n}$ and to define, for every $k$, $X_k=\Phi(U,Y_k)$, where $U$ is a random variable independent on $(Y_k)_{1\leqslant k\leqslant n}$ and $\Phi$ a measurable function.

And, in some (loose) sense, this is the only way... as shown by Diaconis and Freedman for finite sequences, using mixtures of urn sequences rather than mixtures of i.i.d. sequences, and earlier, by de Finetti for infinite sequences.

If the converse were true, then "i.i.d." and exchangeability would be the same thing, and instead of having a separate concept, "exchangeability", one would have a theorem saying there are these two equivalent characterizations of the concept.

Exchangeability is stronger than identical distribution. Note that $X_1\overset{d}{=}X_{j_1}$, so these two are identically distributed, and $j_1$ could be any index.

A couple of important examples of exchangeability:

• First choose $p\in(0,1)$ randomly and suppose it's uniformly distributed, then let $X_1,X_2,X_3,\ldots$ be conditionally i.i.d. given $p$, with $X_1 = \left.\begin{cases} 1 & \text{with probability }p \\ 0 & \text{with probability }1-p \end{cases} \right\}$. Then $X_1,X_2,X_3,\ldots$, which are conditionally independent given the value of $p$, are not independent. To see that, suppose you observe the first 20 of these are equal to $1$. That makes it quite probable that $p$ is near $1$, so that it's likely that $X_{21}$ is equal to $1$. But by symmetry in the way the distribution was described, they are exchangeable. De Finetti's theorem says every infinitely long exchangeable sequence is a mixture of i.i.d. sequences in this manner (but the distribution of $p$ need not be uniform).
• Suppose an urn contains $10$ red and $15$ green marbles, and $X_k$ the number of red marbles on the $k$th draw without replacement, for $k=1,\ldots,12$. If you see $10$ red marbles in the first $10$ draws, then you know the $11$th one will be green, so they're not independent. But it's not hard to show they're exchangeable. Unlike the previous example, this one cannot be extended to an infinitely long sequence of exchangeable random variables.

You can't say anything about the distribution of $X_j$ if you know it's part of an exchangeable sequence, since any distribution at all can be in that role.