Rational Solutions to $\frac{1}{x} = x - \left \lfloor{x}\right \rfloor$

assume there is a solution $x=p/q $ with $x $ irreductible. then $x-1/x $ is an integer. or

$p/q-q/p=(p^2-q^2)/pq$ integer.

thus $p|p^2-q^2$

and $p|q $ which is not possible.

You are on the right track. Continue your argument and you will find the answer.

Let $p=q*n+r $, where $q, n , r \in \mathbb N$. Now we have $\frac{q}{qn+r}=\frac {r}{q}$, and it follows $q^2-r^2=nqr$. Further, $\frac {q}{r}-\frac{r}{q}=n$. Let $\frac {q}{r}=t$ and this leads to a quadric equation w.r.t t, $t^2-tn-1=0$.

Clearly, there is no rational t when $n\in \mathbb N$ since $n^2+4$ can not be a square ($n \neq 0$). In other words, $\frac {q}{r}$ is irrational. This contradicts that $q, r \in \mathbb N$