How to denote matrix concatenation?

There is such a thing as "augmenting" two matrices. For example, augmenting your matrices $A$ and $B$ above gives $$(A\mid B)=\left(\begin{array}{ll}a_1&a_2\\a_3&a_4\end{array}\left|\begin{array}{ll}b_1&b_2\\b_3&b_4\end{array}\right.\right).$$ This is useful notation for Gaussian row reduction since it makes clear the two matrices.

You can also write $[\begin{array}{l}A& B\end{array}]$.

In the example given, $$ C = \begin{bmatrix} A \\ B \end{bmatrix} $$ is a standard notation. This is a particular example of "block notation".