Why should we expect the connection between complex arithmetic and geometry?

We can start way earlier to get a geometric interpretation, at the real numbers. Multiplication by a real number is a combination of scaling and mirroring. Multiplying by a a positive number is scaling the real line, multiplying by $-1$ is mirroring it at the origin. On an abstract level, a core feature of mirroring is that doing it twice returns the original picture. This gives rise to the interpretation that multiplication by $-1$ is a mirroring, since $(-1)^2=1$, so multiplying by $-1$ twice is the identity.

The complex numbers give rise to a similar interpretation. We can still view multiplication by $-1$ as mirroring the plane at the origin, but in a 2d context, we can also see it as a $180^\circ$ rotation. They are really the same. But we also get a new element, $\mathrm i$. Its basic feature is that $\mathrm i^2=-1$, that is, multiplying by $\mathrm i$ twice is rotation by $180^\circ$. But that's also a core feature of rotation by $90^\circ$: rotating by that amount twice is the same as rotating by $180^\circ$ once. So that's a good hint that complex multiplication can have something to do with rotations. We just need to find a fitting topology (a scalar product to describe angles, most importantly) which makes multiplication by $\mathrm i$ an actual $90^\circ$ rotation. And it turns out that the scalar product wrt which $1$ and $\mathrm i$ form an orthonormal basis does just that. So it's a good idea to choose those as a basis of $\mathbb C$ as a real vector space, making them span the coordinate axes. In this picture, multiplication by $\mathrm i$ will be guaranteed to be a $90^\circ$ rotation. And using some algebra, all other complex multiplications can then be shown to also be rotations and scalings.

From the point of view of group theory there is a deep reason : the group of similitudes ( ratio-of-lengths preserving maps) of a (Euclidean) plane is isomorphic to the group of affine (or anti-affine) transformation of a complex line $(z\to az+b$ or $z\to a \bar z+b$). This (exceptional) isomorphism enables us to do geometry by using complex numbers.

This is even more clear if we go to the projective line (the Riemann Sphere). The group of projective transformations of a projective line $PGL(2,C)$ is isomorphic to Möbius group of conformal maps of a sphere $PSO(3,1)$.

You can arrive at complex arithmetic from geometric intuition if you start with transformations of the plane.

It is well known that matrices which preserve angles (i.e. map shapes to similar shapes) and orientation are of the form $cR(\theta)$, where $c$ is a positive number and $R(\theta)$ is a rotation matrix. That is, $$cR(\theta) = c\pmatrix{\cos \theta & -\sin \theta\\ \sin \theta & \cos \theta}. $$

Since $c$ and $\theta$ are arbitrary, these are all matrices of the form $$\pmatrix{a & -b \\ b & a}$$ for $a, b \in \mathbb R$ (except for the null matrix).

Now, after adjoining the null matrix, this set becomes a vector space of dimension two, closed under matrix multiplication, and where all non-null elements have a multiplicative inverse ($c^{-1} R(-\theta)$).

The interesting part is that we can choose a basis like this: $$ \pmatrix{a & -b \\ b & a} = a \pmatrix{1 & 0 \\ 0 & 1} + b \pmatrix{0 & -1 \\ 1 & 0} = a I + b J,$$ where $I$ is the identity matrix and $J=\pmatrix{0 & -1 \\ 1 & 0}$ is a matrix which, under matrix multiplication, has the property $J^2=-I$. That is, it is in some sense the "square root" of $-I$. It also represents rotation in 90º (like the complex $i$ does). Indeed: $J = R(\pi/2)$ and, as expected, $J e_1 = e_2$ and $J e_2 = -e_1$.

Moreover, if you work out the product rule, it is exactly the one that arises in complex numbers:

$$ \pmatrix{a & -b \\ b & a} \cdot \pmatrix{c & -d \\ d & c} \\ = \pmatrix{ac-bd & -(ad+bc) \\ ad+bc & ac-bd} \\ = (ac-bd)I + (ad+bc)J.$$

Furthermore, we can define subtraction, division, and all arithmetic operations for them in a way parallel to how they are defined for complex numbers.

Finally, add to this that the subspace generated by $I$ is an algebraic copy of $\mathbb R$, so you can view the full space as an extension of $\mathbb R$.

To sum up

• Angle and orientation preserving linear transformations carry great geometric meaning (similarity).
• They form a two-dimensional space, which you can think of as an algebraically compatible extension of $\mathbb R$.
• They have two components, one in the direction of the identity/unit and one in the direction of a $\pi/2$ rotation.
• This space can be constructed from $\mathbb R$ just by adjoining an outside element $J$ such that $J^2$ is minus the identity (and extending the usual algebraic rules).
• This is essentially the same recipe as the one for constructing the complex numbers.