Finding an equation of circle which passes through three points

Suppose the points are $A,B,C$. Then intersect the equations of perpendicular bisectors of $AB$ and $BC$. This is the center of the desired circle. (with your notation $(p,q)$)

Now calculate the distance between $(p,q)$ and $A$. Now $r$ is also found.

$\begin{vmatrix} x^2+y^2&x&y&1\\ 5^2+10^2&5&10&1\\ (-5)^2+0^2&-5&0&1\\ 9^2+(-6)^2&9&-6&1\\ \end{vmatrix}= \begin{vmatrix} x^2+y^2&x&y&1\\ 125&5&10&1\\ 25&-5&0&1\\ 117&9&-6&1\\ \end{vmatrix} = 0$

I know i need to use that formula but have no idea how to start

\begin{equation*} \left( x-q\right) ^{2}+\left( y-p\right) ^{2}=r^{2}\tag{0} \end{equation*}

A possible very elementary way is to use this formula thrice, one for each point. Since the circle passes through the point $(5,10)$, it satisfies $(0)$, i.e.

$$\left( 5-q\right) ^{2}+\left( 10-p\right) ^{2}=r^{2}\tag{1}$$

Similarly for the second point $(-5,0)$:

$$\left( -5-q\right) ^{2}+\left( 0-p\right) ^{2}=r^{2},\tag{2}$$

and for $(9,-6)$:

$$\left( 9-q\right) ^{2}+\left( -6-p\right) ^{2}=r^{2}.\tag{3}$$

We thus have the following system of three simultaneous equations and in the three unknowns $p,q,r$:

$$\begin{cases} \left( 5-q\right) ^{2}+\left( 10-p\right) ^{2}=r^{2} \\ \left( -5-q\right) ^{2}+p^{2}=r^{2} \\ \left( 9-q\right) ^{2}+\left( 6+p\right) ^{2}=r^{2} \end{cases}\tag{4} $$

To solve it, we can start by subtracting the second equation from the first

$$\begin{cases} \left( 5-q\right) ^{2}+\left( 10-p\right) ^{2}-\left( 5+q\right) ^{2}-p^{2}=0 \\ \left( 5+q\right) ^{2}+p^{2}=r^{2} \\ \left( 9-q\right) ^{2}+\left( 6+p\right) ^{2}=r^{2} \end{cases} $$

Expanding now the left hand side of the first equation we get a linear equation

$$\begin{cases} 100-20q-20p=0 \\ \left( 5+q\right) ^{2}+p^{2}=r^{2} \\ \left( 9-q\right) ^{2}+\left( 6+p\right) ^{2}=r^{2} \end{cases} $$

Solving the first equation for $q$ and substituting in the other equations, we get

$$\begin{cases} q=5-p \\ \left( 10-p\right) ^{2}+p^{2}-\left( 4+p\right) ^{2}-\left( 6+p\right) ^{2}=0 \\ \left( 4+p\right) ^{2}+\left( 6+p\right) ^{2}=r^{2} \end{cases} $$

If we simplify the second equation, it becomes a linear equation in $p$ only

$$\begin{cases} q=5-p \\ 48-40p=0 \\ \left( 4+p\right) ^{2}+\left( 6+p\right) ^{2}=r^{2} \end{cases} $$

We have reduced our quadratic system $(4)$ to two linear equations plus the equation for $r^2$. From the second equation we find $p=6/5$, which we substitute in the first and in the third equations to find $q=19/5$ and $r^2=1972/25$, i.e

$$\begin{cases} q=5-\frac{6}{5}=\frac{19}{5} \\ p=\frac{6}{5} \\ r^{2}=\left( 4+\frac{6}{5}\right) ^{2}+\left( 6+\frac{6}{5}\right) ^{2}= \frac{1972}{25}. \end{cases}\tag{5} $$

So the equation of the circle is

\begin{equation*} \left( x-\frac{19}{5}\right) ^{2}+\left( y-\frac{6}{5}\right) ^{2}=\frac{1972}{25}. \end{equation*}