Intermediate ring between a field and an algebraic extension.

Well, you know that $R$ is contained in an algebraic extension of $F$, so you should use it somehow. It directly implies that there is a polynomial (of minimal degree, say) with coefficients in $F$ which annihilates $r$. Can you manufacture an inverse for $r$ using this polynomial?

Let $F\subseteq R\subseteq E$, with $E$ algebraic over $F$, $R$ a ring.

As you note, the key is to show that any nonzero element of $r$ has an inverse in $R$.

So let $r\in R$, $r\neq 0$. Since $R\subseteq E$, then $r$ is algebraic over $F$. Therefore, there is a monic irreducible polynomial $p(x)$ with coefficients in $E$ with $p(r)=0$. Write $$p(x) = x^n + a_{n-1}x^{n-1}+\cdots + a_1x + a_0.$$ Note that $a_0\neq 0$, since we are taking $p(x)$ to be the monic irreducible. We have $$\begin{align*} p(r) &= 0\\ r^n + a_{n-1}r^{n-1}+\cdots + a_1r + a_0 & = 0\\ r^n + a_{n-1}r^{n-1}+\cdots +a_1r &= -a_0\\ r(r^{n-1}+a_{n-1}r^{n-2}+\cdots + a_1) &= -a_0\\ r\left(-\frac{1}{a_0}\right)(r^{n-1}+a_{n-1}r^{n-2}+\cdots + a_1) &= 1. \end{align*}$$

Hint $\ $ Read off an inverse of $\rm\:r \ne 0\:$ from a minimal polynomial $\rm\:f(x)\in F[x]\:$ of $\rm\:r\:$ over $\rm\:F,\,$ e.g. $\rm\,ax^2+bx = c \,\Rightarrow\, x\,(ax+b)/c = 1,\,$ where $\rm\,c\neq 0\,$ by minimality (else we could cancel $\rm x)$

Note: this is a special case of using the Euclidean algorithm to compute inverses via the Bezout identity, viz. $\rm\: (x,f(x)) = 1\ \Rightarrow\ a(x)\ x + b(x)\ f(x) = 1\ $ so $\rm\: a(r)\ r = 1\ $ by evaluating at $\rm\:x = r.\:$

This can be viewed as generalization of rationalizing denominators in low-degree extensions, i.e. one may compute an inverse of $\rm\:r\:$ by "rationalizing" (to $\rm\:F)\,$ the denominator of $\rm\:1/r\:$ (which can also be done using norms, resultants, etc). See here for much further discussion.