Does the expression $x^4 +y^4$ take on all values in $\mathbb{Z}/p\mathbb{Z}$?

Expanding on a comment, the curve $X^4+Y^4=aZ^4$ (for $a\ne0$) has genus $3$. So the Hasse-Weil bound says $$ N_p(a) := \#\bigl\{ [X,Y,Z]\in\mathbb P^2(\mathbb F_p) : X^4+Y^4=aZ^4 \bigr\} $$ satisfies $$ \bigl| N_p(a) - p - 1 \bigr| \le 2g\sqrt{p} = 6\sqrt{p}. $$ Thus $$ N_p(a) \ge p + 1 - 6\sqrt{p}. $$ There are at most $4$ points with $Z=0$, so $$ \#\bigl\{ (X,Y) \in\mathbb A^2(\mathbb F_p) : X^4+Y^4=a \bigr\} \ge p-3-6\sqrt{p}. $$ So you'll always have a solution provided $p\ge3+6\sqrt{p}$, which means that there is always a solution provided $p\ge43$.

emtom has found the right reference, but there is a more explicit result in that book (Ireland and Rosen, A Classical Introduction to Modern Number Theory). In fact, Theorem 5 of Chapter 8 (on page 103) directly implies that the number $N = N_{p,\alpha}$ of solutions to $x^4+y^4=\alpha$ in $\mathbb{F}_p$ satisfies the inequality $$ \left| N - p \right| \leq M_0 + M_1 p^{1/2} $$ for some $M_0$ and $M_1$ that are described explicitly in the statement of the theorem (and from that description it easily follows they can be bounded in a way that is independent of $p$ or $\alpha$).

It then automatically follows that for sufficiently large $p$, we will have $N_{p,\alpha}>0$ for all $\alpha$. In other words, for sufficiently large $p$, the expression $x^4+y^4$ assumes all values of $\mathbb{F}_p$ as $x$ and $y$ run through $\mathbb{F}_p$.

Of course, this is no better (and possibly slightly worse) than what you get from the Hasse-Weil bound which Dan Loughran referred to, but at least this reference has the virtue of providing a completely elementary proof.

Not exactly an answer, but in exercise $18$ in page $106$ of Ireland and Rosen's A Classical Introduction to Modern Number theory it states: Let $p\equiv 1\mod 4$ and let $p=A^2+B^2$ where we fix $A$ by requiring that $A\equiv 1\mod 4$. Then $N=\#\{(x,y)\in\mathbb{F}_p^2\mid x^4+y^4=1\}$ satisfies $N=p-3-6A$ if $p\equiv1\mod 8$ and $N=p+1+2A$ if $p\equiv5\mod 8$.