Why is circular motion circular?
Since speed is constant, we can write the velocity as $$\vec{v} = \begin{bmatrix}v_x(t) \\ v_y(t)\end{bmatrix} = v\begin{bmatrix}\cos(\theta_v(t)) \\ \sin(\theta_v(t))\end{bmatrix}$$ for some unknown function of time $\theta_v(t)$. This is because $$|\vec{v}| = v = \sqrt{v_x^2 + v_y^2} = const.$$
Similarly, for a constant magnitude force $$\vec{a} = \vec{F}/m = a\begin{bmatrix}\cos(\theta_a(t)) \\ \sin(\theta_a(t))\end{bmatrix}$$ for another unknown function $\theta_a(t).$
With the force perpendicular to velocity and with $\vec{v}$ and $\vec{a}$ non-zero, we have $$\vec{a}\cdot\vec{v} = av\left(\cos(\theta_a(t))\cos(\theta_v(t)) + \sin(\theta_a(t))\sin(\theta_v(t))\right) = 0$$ $$\cos(\theta_a(t))\cos(\theta_v(t)) + \sin(\theta_a(t))\sin(\theta_v(t)) = 0.$$ This expression simplifies to $$\cos(\theta_a(t) - \theta_v(t)) = 0,$$ which implies that $$\theta_a(t) - \theta_v(t) = \left(n + \frac{1}{2}\right)\pi$$ for some integer $n$. So, $$\cos(\theta_a(t)) = \cos\left(\left(n + \frac{1}{2}\right)\pi + \theta_v(t)\right) = -\sin(\theta_v(t))$$ $$\sin(\theta_a(t)) = \sin\left(\left(n + \frac{1}{2}\right)\pi + \theta_v(t)\right) = \phantom{-}\cos(\theta_v(t))$$
From the definition of acceleration, we have \begin{align} \vec{a} &= \frac{d\vec{v}}{dt} \\ a\begin{bmatrix}\cos(\theta_a(t)) \\ \sin(\theta_a(t))\end{bmatrix} &= v\frac{d\theta_v}{dt}\begin{bmatrix}-\sin(\theta_v(t)) \\ \phantom{-}\cos(\theta_v(t))\end{bmatrix}. \end{align} Using the $\cos$ and $\sin$ relations above, \begin{align} a\begin{bmatrix}-\sin(\theta_v(t)) \\ \phantom{-}\cos(\theta_v(t))\end{bmatrix} &= v\frac{d\theta_v}{dt}\begin{bmatrix}-\sin(\theta_v(t)) \\ \phantom{-}\cos(\theta_v(t))\end{bmatrix}. \end{align} Thus, $$a = v\frac{d\theta_v}{dt}$$ Since, $a$ and $v$ are both constant, $d\theta_v/dt$ must also be constant. $$\frac{d\theta_v}{dt} = \omega$$ $$\theta_v(t) = \omega t + \theta_0$$ for constant $\theta_0$ and $\omega = a/v$.
Substituting back into the original $\vec{v}$ equation results in $$\vec{v} = v\begin{bmatrix}\cos(\omega t + \theta_0) \\ \sin(\omega t + \theta_0)\end{bmatrix}.$$ Integrating yields $$\vec{x} = \frac{v}{\omega}\begin{bmatrix}\phantom{-}\sin(\omega t + \theta_0) \\ -\cos(\omega t + \theta_0)\end{bmatrix} + \begin{bmatrix}x_0 \\y_0\end{bmatrix}$$ which is circular motion about the point $(x_0, y_0)$ with a radius of $$r = \frac{v}{\omega} = \frac{v^2}{a} = \frac{mv^2}{F}.\qquad\left(\textrm{using }\omega=\frac{a}{v}\right)$$ This gives us the formula for centripetal force $$F = \frac{mv^2}{r}$$ and acceleration $$a = \frac{v^2}{r}.$$
Addendum, the first: The following assumptions are redundant:
- Speed is constant.
- Force is perpendicular to velocity.
Since $$\frac{d(|\vec{v}|^2)}{dt} = \frac{d(\vec{v}\cdot\vec{v})}{dt} = 2\vec{v}\cdot\frac{d\vec{v}}{dt} = 2\vec{v}\cdot\vec{a}$$ So, if (and only if) speed is constant, then the acceleration vector will be zero or perpendicular to the velocity vector.
Addendum the second: This derivation is only valid in 2D. In 3D, you can add a velocity component that is perpendicular to the original velocity and the force (a constant $v_z$, for example). This results in helical motion. Charged particles in a uniform magnetic field do this. The magnetic force is $\vec{F} = q\vec{v} \times \vec{B}$, which is always perpendicular to the velocity of the particle. In fact, as long as the magnetic field is of constant magnitude, the force will be of constant magnitude, and the particle will follow a helical path around a curved line that follows the curved magnetic field lines.
I am not giving a general rigorous proof (so basically, I'm not answering your question), but rather, a simple, semi-intuitive illustration (so basically, I am shedding some alternative mathematical light onto this). The logic would be as follows:
In the case of a constant velocity perpendicular to a constant force, the force can not do any work. This can be seen as the infinitesimal amount of work done in producing a small displacement $d{\vec r}$ would read $$dW = {\vec F} \cdot d{\vec r} = {\vec F} \cdot {\vec v} \ dt = 0,$$ as ${\vec F} \cdot {\vec v} = 0$ owing to the two being perpendicular.
Work-Energy Theorem tells us that no work done $\Rightarrow$ no change in energy. Thus, the point particle subjected to these conditions (mutually perpendicular force and velocity, both of constant strength), would move in a manner such that its energy stays constant.
Since the energy expression depends on the magnitude of ${\vec v}$, and not explicitly on its direction, if only the direction of ${\vec v}$ changes, and not its magnitude, energy stays the same. Thus, the above conditions would stand respected in this case, even with the continuous motion of the point particle.
The problem is thus mathematically down to ** finding the locus of this moving point particle, such that its velocity remains the same in magnitude, but keeps changing direction?**
Clearly uniform circular motion is a special case where this is true. The whole of central force motion, i.e. elliptical orbits (cf. Toby Peterken's answer ) is another situation where this holds.
So clearly, uniform circular motion is not a unique solution of this problem.