Proof that a trigonometric function of a rational angle must be non-transcendental

To start with, let's notice that we might just as well prove this fact for the cosines of rational multiples of $\pi$ as the sines. We just need to use the identity $\sin x=\cos\left(\frac{\pi}{2} - x\right)$, which gives us: $$\sin \frac{a\pi}{b}=\cos\left(\frac{\pi}{2} - \frac{a\pi}{b}\right)=\cos\left(\frac{(b-2a)\pi}{2b}\right)$$ As this is the cosine of a rational multiple of $\pi$, we get the result for sines more or less for free from the result for cosines.

Once this preliminary is out of the way, the key thing to notice is that $\cos(nx)$ can always be written as a polynomial function of $\cos x$. For example: \begin{align} \cos 2x &= 2 \cos^2 x - 1 \\ \cos 3x &= 4 \cos^3 x - 3 \cos x\\ \cos 4x &= 8 \cos^4 x - 8 \cos^2 x + 1 \\ \cos 5x &= 16 \cos^5 x - 20 \cos^3 x + 5 \cos x \end{align} and so on. (Formulas taken from here.)

Why does this matter? It means that we can find an explicit polynomial which has $\cos \frac{a\pi}{b}$ as a root. To see an example of this, let's look at $\alpha = \cos \frac{2\pi}{5}$. Using the last formula above, we know that $$ \cos (2\pi)=\cos\left(5 \cdot \frac{2\pi}{5}\right)=16 \cos^5 \frac{2\pi}{5}- 20 \cos^3 \frac{2\pi}{5} + 5 \cos \frac{2\pi}{5}=16\alpha^5-20\alpha^3+5\alpha $$ Since $\cos(2\pi)=1$, we know that $\alpha$ is a solution to the equation $16x^5-20x^3+5x=1$, or, to put it another way, a root of the polynomial $16x^5-20x^3+5x-1$. So, by definition, $\alpha$ is algebraic (i.e., not transcendental).

You can use multiple-angle formulas to do this in general. If you want to prove that $\beta=\cos\frac{a\pi}{b}$ is algebraic, you just need to use the formula for $\cos bx$. This will give you a polynomial expression for $\cos(a\pi)$ in terms of $\beta$; since we know that $\cos(a\pi)=\pm 1$, that's enough to prove that $\beta$ is algebraic.

So we just need to prove this key fact, which can be done by induction. For the base case, notice that $\cos x$ and $\cos 2x=2\cos^2 x -1$ are both polynomial functions of $\cos x$. For the inductive step, we'll suppose that $\cos nx$ and $\cos (n-1)x$ are polynomials, and use the identity given here for $\cos a + \cos b$, with $a=(n+1)x$, $b=(n-1)x$:

$$\cos(n+1)x + \cos(n-1)x=2\cos nx \cos x$$

This can be rearranged into the formula $\cos(n+1)x = 2 \cos x \cos nx - \cos(n-1)x$, which means that if $\cos nx$ and $\cos(n-1)x$ are polynomials in $\cos x$, then so is $\cos(n+1)x$. So the proof by induction is finished.

If you're interested in knowing more details about this kind of polynomial formula for $\cos nx$ in terms of $\cos x$, a useful keyword to search on would be "Chebyshev polynomials."

As a final note, we could try to do this proof directly for $\sin \frac{a\pi}{b}$, as there are multiple-angle formulas for $\sin$ which are similar to the ones for $\cos$. But it'd be a little bit trickier, because those formulas often involve both $\sin$ and $\cos$. For example, $\sin 2x=2\cos x \sin x$, and getting rid of the $\cos$ would involve taking a square root and doing a bunch of annoying case analysis involving signs.

Write:

$$z=e^{a\pi i/b}=\cos(a\pi/b)+i\cdot\sin(a\pi/b)$$

Then,

$$z^{2b}=e^{2a\pi i}=(e^{2\pi i})^{a}=1^a=1$$

$z$ is an algebraic number, as a root of the equation $x^{2b}-1=0$. Thus its real and imaginary parts ($\cos(a\pi/b)$ and $\sin(a\pi/b)$ respectively) are both algebraic.

For each positive integer $b,$

\begin{align} \cos b\theta &= \frac{e^{ib\theta}+e^{-ib\theta}}{2} \\&= \frac{(\cos\theta + i \sin \theta)^b + (\cos \theta - i \sin \theta)^b}{2} \\&= \frac{\sum_{k=0}^b \binom{b}{k} (\cos^{b-k} \theta) (i^{k})(\sin^{k}\theta)+\sum_{k=0}^b\binom{b}{k} (\cos^{b-k} \theta) (-1)^{k}(i^{k})(\sin^{k}\theta)}{2} \\&= \frac{\sum_{\substack{0\le k\le b \\k \text{ even}}} 2 \binom{b}{k} (\cos^{b-k} \theta) ((-1)^{k/2})(\sin^{k}\theta)}{2} \\&= \sum_{\substack{0\le k\le b \\k \text{ even}}}(-1)^{k/2} \binom{b}{k} \cos^{b-k}\theta \sin^{k}\theta \\&= \sum_{\substack{0\le k\le b \\k \text{ even}}}(-1)^{k/2} \binom{b}{k} \cos^{b-k} \theta (1-\cos^{2}\theta)^{k/2}. \end{align}

Write $P_b(x) = \sum_{\substack{0\le k\le b \\k \text{ even}}}(-1)^{k/2} \binom{b}{k} x^{b-k} (1-x^2)^{k/2}.$

$P_b$ is a polynomial in one variable with integer coefficients, with the property that

$$\cos(b\theta) = P_b(\cos \theta)$$

for all $\theta.$

Now, if $\theta = a \pi/b$ where $a$ is an integer and $b$ is, without loss of generality, a positive integer, then

$$P_{b}(\cos \theta) = \cos (a \pi) = (-1)^a.$$

$P_{b}(x)-(-1)^a$ is not identically zero (since, if it were, the identity $P_{b}(\cos \theta)=\cos(b \theta)$ could not hold).

It follows that $\cos (\frac{a}{b}\pi) = \cos \theta$ is algebraic, since it's the root of a non-zero polynomial with integer coefficients.

Finally, $\sin (\frac{a}{b}\pi) = \cos (\frac{a}{b} \pi - \frac{\pi}{2}) = \cos(\frac{2a-b}{2b} \pi),$ which we already know is algebraic by the argument above, since $2a-b$ and $2b$ are integers.

If you want to exhibit a specific polynomial with integer coefficients that $\sin (\frac{a}{b}\pi)$ is a root of, $P_{2b}(x)-(-1)^b$ will work.