Common tangent to two circles

A line with equation $ax+y+c=0$ is tangent to a circle iff its distance from the center of the circle is equal to the radius of the circle, so you get the system of equations:

$$\left\{\begin{array}{rcl}|2a+c| &=& 3\sqrt{a^2+1} \\ |5a+4+c| &=& 2\sqrt{a^2+1}. \end{array}\right.$$

with $4$ solutions.

You can approach this with homogeneous coordinates. A circle with equation $(x-x_c)^2+(y-y_c)^2 - r^2 = 0$ is represented by a 3x3 matrix as

$$ {\rm Circle}(x_c,y_c,r) = \begin{bmatrix} 1 & 0 & -x_c \\ 0 & 1 & -y_c \\ -x_c & -y_c & x_c^2+y_x^2-r^2 \end{bmatrix} $$

This means that the equation for a circle $C_1 = \begin{bmatrix} 1 & 0 & -2 \\ 0 & 1 & 0 \\ -2 & 0 & -5 \end{bmatrix}$ is given by the quadratic form

$$ P^\top C_1 P = 0 $$ $$ x^2-4 x+y^2-5 = 0 $$

which is the equation for the first circle when expanded out, and $P=\begin{pmatrix} x&y&1 \end{pmatrix} ^\top $ is an arbitrary point.

The second circle is $ C_2 = \begin{bmatrix} 1 & 0 & -5 \\ 0 & 1 & -4 \\ -5 & -4 & 37 \end{bmatrix} $.

Now here is the fun stuff. A line in this notation in general is defined as $L=\begin{vmatrix}a&b&c\end{vmatrix}^\top$ such that the equation of the line is

$$ P^\top L =0 $$ $$ a x+b y+c = 0 $$

Actually $a$, $b$ above designate the direction of the line such that if the line makes an angle $\theta$ with the horizontal then the line is $L=\begin{vmatrix}-\sin\theta&\cos\theta&-d\end{vmatrix}^\top$ and $d$ is the distance of the line to the origin.

We are using the above information to find the lines that are tangent to both circles. A tangent line to the first circle has satisfies the equation

$$ L^\top C_1^{-1} L =0 $$ $$ d = \pm 3 -2 \sin \theta $$

with the two possible line equations

$$ L_A = \begin{vmatrix}-\sin\theta_A&\cos\theta_A &2\sin\theta_A-3\end{vmatrix}^\top $$ $$ L_B = \begin{vmatrix}-\sin\theta_B&\cos\theta_B &2\sin\theta_B+3\end{vmatrix}^\top $$

to find the orientations of these lines $\theta_A$ and $\theta_B$ we have to find the lines that are tangent to the second circle, and match the coefficients

$$ L^\top C_2^{-1} L =0 $$ $$ d = \pm 2 + 4 \cos \theta -5 \sin \theta $$

with also two possible line equations

$$ L_A = \begin{vmatrix}-\sin\theta_A&\cos\theta_A &-4\cos\theta_A+5\sin\theta_A-2\end{vmatrix}^\top $$ $$ L_B = \begin{vmatrix}-\sin\theta_B&\cos\theta_B &-4\cos\theta_B+5\sin\theta_B+2\end{vmatrix}^\top $$

Setting $L_A=L_A$ and solving for $\theta_A$ yields the following

$$ 2\sin\theta_A-3 = -4\cos\theta_A+5\sin\theta_A-2 $$ $$ 4\cos\theta_A-3\sin\theta_A =1 $$ $$ \sin\theta_A = \frac{8\sqrt{6}-3}{25} $$

with the solution

$$ L_A = \begin{vmatrix} \frac{3-8\sqrt{6}}{25} & \frac{4+6\sqrt{6}}{25} & \frac{16 \sqrt{6}-81}{25} \end{vmatrix} $$ $$ \left( \frac{3-8\sqrt{6}}{25}\right) x + \left(\frac{4+6\sqrt{6}}{25}\right) y + \left(\frac{16 \sqrt{6}-81}{25}\right) = 0 $$ $$ -0.664 x + 0.7479 y - 1.6723 = 0 $$

Similarly with $L_B=L_B$ yielding $4\cos\theta_B-3\sin\theta_B=-1$ or

$$ L_B = \begin{vmatrix} -\frac{3+8\sqrt{6}}{25} & \frac{6\sqrt{6}-4}{25} & \frac{16 \sqrt{6}+81}{25} \end{vmatrix} $$ $$ -\left(\frac{3+8\sqrt{6}}{25}\right) x + \left(\frac{6\sqrt{6}-4}{25}\right) y + \left(\frac{16 \sqrt{6}+81}{25}\right) = 0 $$ $$ -0.904 x + 0.4278 y + 4.808 = 0 $$

Here are the results plotted in GeoGebra for validation.

The parametric equation of $(x-a)^2+(y-b)^2=r^2$ is $(x=a+r\cos C,y=b+r\sin C)$

Let $$\frac{y-(b+r\sin C)}{x-(a+r\cos C)}=m$$ be a tangent at $(a+r\cos C,b+r\sin C)$, then the distance of line from the center is equal to the radius.

$$r=\frac{\mid m(a-a-r\cos C)-b+b+r\sin C\mid}{\sqrt{m^2+1}}=\frac{r\mid\sin C-m\cos C\mid}{\sqrt{m^2+1}}$$

$m^2+1=(\sin C-m\cos C)^2$

$\implies (m\sin C+\cos C)^2=0\implies m=-\frac{\cos C}{\sin C}$

So, the equation of the tangent becomes

$$\frac{y-(b+r\sin C)}{x-(a+r\cos C)}=-\frac{\cos C}{\sin C}$$

$x\cos C+y\sin C-a\cos C-b\sin C -r=0$ (this can also be reached using calculus)

For $(x-2)^2+y^2=9,a=2,b=0,r=3$

So, equation of the tangent will be $x\cos A+y\sin A-2\cos A -3=0$

For $(x-5)^2+(y-4)^2=4,a=5,b=4,r=2$

So, equation of the tangent will be $x\cos B+y\sin B-5\cos B-4\sin B -2=0$

For common tangent, these two lines must be same,

So, $$\frac{\cos A}{\cos B}=\frac{\sin A}{\sin B}=\frac{2\cos A+3}{5\cos B+4\sin B+2}$$

$\frac{\cos A}{\cos B}=\frac{\sin A}{\sin B}\implies \sin(A-B)=0$

$\implies A=B$ or $A=\pi+B$

(1)If $A=B,1=\frac{2\cos B+3}{5\cos B+4\sin B+2}\implies 4\sin B+3\cos B=1$

(2)If $A=\pi+B,\cos A=\cos(\pi+B)=-\cos B, -1=\frac{-2\cos B+3}{5\cos B+4\sin B+2}$

For (1), $4\sin B+3\cos B=1$

Putting $4=R\sin D,3=R\cos D\implies R=5,D=cos^{-1}\frac 3 5,$

$\cos(B-D)=\frac 1 5, B-\cos^{-1}\frac 3 5 =2n\pi \pm \cos^{-1}\frac 1 5$ where $n$ is any integer.

$\cos B=\frac{3 \pm 8\sqrt{6}}{25},\sin B$ can be calculated uniquely using (1).

So, there will be two tangent in this case.

For(2) $ 3\cos B+4\sin B=-5$

Applying the same approach like in (1), $\cos(B-\cos^{-1}\frac 3 5)=-1=\cos \pi, B=\cos^{-1}\frac 3 5+\pi$ $\cos B=\cos(\cos^{-1}\frac 3 5+\pi)=-\cos(\cos^{-1}\frac 3 5)=-\frac 3 5$

$\sin B$ becomes $-\frac 4 5$

So, the tangent becomes, $(x-5)(-\frac 3 5)+(y-4)(-\frac 4 5) -2=0$