Switching order of supremum for doubly indexed sequence?

Yes, it means exactly what you wrote.

The reason that

$$\sup_i\,\sup_j\ \alpha_{ij}=\sup_j\,\sup_i\ \alpha_{ij}$$

is that they're both equal to

$$\sup_{i,j}\,\alpha_{ij}\;.$$

For assume that

$$\sup_i\,\sup_j\ \alpha_{ij}\lt\sup_{i,j}\,\alpha_{ij}\;.$$

Then $\sup_i\,\sup_j\ \alpha_{ij}$ is not an upper bound for the $\alpha_{ij}$ (since there is no upper bound less than the supremum). Thus there is some $\alpha_{kl}$ greater than $\sup_i\,\sup_j\ \alpha_{ij}$. But this $\alpha_{kl}$ would make $\sup_j\alpha_{kj}$ be at least $\alpha_{kl}$, and thus $\sup_i\,\sup_j\alpha_{ij}$ would also be at least $\alpha_{kl}$, a contradiction.

Similarly, if

$$\sup_i\,\sup_j\ \alpha_{ij}\gt\sup_{i,j}\,\alpha_{ij}\;,$$

then some $\alpha_{kl}$ would have to be greater than $\sup_{i,j}a_{ij}$, which is impossible.

Note that this only works because both operations are suprema. If you take, say, the infimum with respect to $i$ and the supremum with respect to $j$, then it does matter in which order you perform those operations.