Can we characterize the Möbius transformations that maps the unit disk into itself?
Consider the function $$f(z)=\frac{e^{i \theta}(z-a)}{1- \bar {a}z}$$ where $a$ is in interior of the disk.
Now we have two parts to prove:
The function $f$ maps unit circle to unit circle and $a$ to $0$.
Every Möbius transformation that preserves the unit disk must be of the above form.
The second statement can be proved by noting that every Möbius transformation is uniquely determined by its action on $3$ points. Try looking at the points $1,0, \infty$.
Most answers seem to be characterizing Möbius transformations which map the unit disk onto itself, which is relatively well-known. If you are asking which map the disk into itself, the article here gives a simple proof that $|z|<1 \Rightarrow |f(z)|<1$ if and only if $$|b\overline{d}-a\overline{c}|+|ad-bc|\leq |d|^2-|c|^2 $$
These transformations make a group, isomorphic to $PSL_2(\mathbb R),$ which takes the upper half plane to itself. The general form, with complex numbers $\alpha, \beta$ and $|\alpha| > |\beta|,$ is $$ f(z) = \frac{\alpha z + \beta}{\bar{\beta} z + \bar{\alpha}}. $$ This is the result of taking real numbers $a,b,c,d$ with $ad-bc > 0$ and calculating $$ \left( \begin{array}{rr} 1 & -i \\ -i & 1 \end{array} \right) \cdot \left( \begin{array}{rr} a & b \\ c & d \end{array} \right) \cdot \left( \begin{array}{rr} 1 & i \\ i & 1 \end{array} \right) = \left( \begin{array}{rr} (a+d) +(b-c)i & (b+c) +(a-d)i \\ (b+c) + (d-a)i & (a+d) + (c-b)i \end{array} \right). $$
We need the modulus of $\alpha$ to be the larger so that $|f(0)| < 1.$ For your own comfort, check that $f(1), f(-1), f(i), f(-i)$ all have modulus $1.$
To get down to three real variables underlying the thing, we may divide through by the positive real number $|\alpha|,$ thereby demanding $\alpha = e^{i \theta}$ have modulus $1,$ then $|\beta| < 1,$ using a new variable $\gamma$ with $|\gamma| < 1$ we have $$ f(z) = \frac{ e^{i \theta} z + \gamma}{\bar{\gamma} z + e^{-i \theta}}. $$