How can we find the derivative of a circle if a circle is not a function?

While each $x\in(-r,\,r)$ is compatible with two choices of $y$, continuous motion along the circumference well-defines the choice of $y$ at each point, giving a local $y$-as-a-function-of-$x$ behaviour wherever $dy/dx$ is finite and nonzero (i.e. $x,\,y$ are both nonzero), which happens at all but four of the circumference's points.

This local behaviour is more easily described in terms of the polar angle $\theta$, and since $x=r\cos\theta,\,y=r\sin\theta$, by the chain rule $\frac{dy}{dx}=\frac{dy/d\theta}{dx/d\theta}=\frac{x}{-y}=-\cot\theta$.

You can still think of it in terms of the slope of a tangent line, and even in terms of a limit. However, as you've pointed out, $x^2 + y^2 = r^2$ isn't a function because it fails the vertical line test. However, by the Implicit Function Theorem we can consider $F(x,y) = x^2 + y^2 - r^2$, and for any $(x_{0},y_{0})$ where $\frac{\partial F}{\partial y}\ne 0$ then there exists some neighborhood around the point $(x_{0},y_{0})$ for which we can express $F(x,y) = 0$ as some function $y = f(x)$. Note that in this case, $$\frac{\partial F}{\partial y} = 2y,$$ which is zero whenever $y = 0$, so at the points $(r,0)$ and $(-r,0)$. On the circle. Anywhere else we can define the curve by either $$y = \sqrt{r^{2} - x^{2}}$$ or $$y = -\sqrt{r^{2} - x^{2}}$$ and these functions are differentiable so long as $y\ne 0$. To verify that the derivative via the limit definition matches that obtained by implicit differentiation, we can compute as follows for the positive semicircle: \begin{align} y' &= \lim_{h\to 0}\frac{\sqrt{r^{2} - (x+h)^{2}} - \sqrt{r^2 - x^2}}{h}\\ &=\lim_{h\to 0}\frac{\sqrt{r^{2} - (x+h)^{2}} - \sqrt{r^2 - x^2}}{h} \cdot \frac{\sqrt{r^{2} - (x+h)^{2}} + \sqrt{r^2 - x^2}}{\sqrt{r^{2} - (x+h)^{2}} + \sqrt{r^2 - x^2}}\\ &=\lim_{h\to 0}\frac{r^{2} - (x+h)^{2} - r^{2} - x^{2}}{h(\sqrt{r^{2} - (x+h)^{2}} + \sqrt{r^2 - x^2})}\\ &=\lim_{h\to 0}\frac{-2xh -h^{2}}{h(\sqrt{r^{2} - (x+h)^{2}} + \sqrt{r^2 - x^2})}\\ &=\lim_{h\to 0}\frac{-2x - h}{\sqrt{r^{2} - (x+h)^{2}} + \sqrt{r^2 - x^2}}\\ &=-\frac{2x}{2\sqrt{r^{2} - x^{2}}}\\ &=-\frac{x}{y}. \end{align} Where the last equality comes from the equation $y = \sqrt{r^2 - x^2}.$ We could do the same for the negative semicircle.