Representation of a group of prime power order
If $G$ is abelian then all irreducible representations have degree $1$, and the result follows. So assume that $G$ is nonabelian.
We show first that there is an abelian normal subgroup $N$ of $G$ of order $p^3$. To see this, let $K$ be a normal subgroup of order $p^2$, so $K$ is abelian. Then $G/C_G(K)$ is isomorphic to a subgroup of ${\rm Aut}(K)$. We have either $K\cong C_{p^2}$ and $|{\rm Aut}(K)| = p(p-1)$, or $K \cong C_p \times C_p$ and $|{\rm Aut}(K)| = p(p^2-1)$. So in either case $|{\rm Aut}(K)|$ is not divisible by $p^2$, and hence $|C_G(K)| \ge p^3$, and we can take $N$ to be a subgroup of $C_G(K)$ of order $p^3$.
Now Clifford's Theorem says that, if $\rho$ is an irreducible representation of $G$ and $N \unlhd G$, then the restriction $\rho_N$ of $\rho$ to $N$ is equivalent to the sum of representations $\rho_1 \oplus \cdots \oplus \rho_k$ for some $k$, where each $\rho_i$ is itself the sum of isomorphic irreducible representations $\rho_{i1} \oplus \cdots \oplus \rho_{it}$ for some $t$, and where $\rho_{ij}$ is equivalent to $\rho_{i'j'}$ iff $i=i'$. Furthermore, the representations $\rho_i$ are permuted transitively by the conjugation action of $G$.
We apply this to our abelian normal subgroup $N$ of order $p^3$, and assume that $\rho$ has degree greater than $1$. Since $N$ is abelian, the $\rho_{ij}$ all have dimension $1$. If $k=1$ then the irreducible constituents of $\rho_N$ are all equivalent, so the image of $\rho_N$ consists of scalar matrices. But then $N/K \le Z(G/K)$, where $K$ is the kernel of the representation. But since the quotient by the centre cannot be cyclic and nontrivial, this implies that $G/K$ is abelian, and hence $\rho$ has degree $1$, contrary to assumption. So $k>1$. Since the $\rho_i$ are permuted transitively by $G$, $k$ must divide $|G| = p^4$, and since $\rho$ has degree less than $p^2$, we must have $k=p$.
But since $|G/N|=p$ and $\rho$ is irreducible, $\rho_N$ cannot have more than $p$ irreducible constituents, and hence $t=1$ and $\rho$ has degree $p$.
So we have proved that all irreducible representations of $G$ have degree $1$ or $p$.
Here is a proof not using Clifford theory. All arguments are taken from Chapter 2 of Isaacs's book on character theory, in particular the following facts (see results 2.27--31 and Exercise 2.9(b) there). $\DeclareMathOperator{\Irr}{Irr}$
Fact 1. Let $\chi$ be a character of a finite group $G$ and $H\leq G$ a subgroup. Then
$$ [\chi_H, \chi_H] \leq \lvert G:H \rvert [\chi,\chi] $$
with equality iff $\chi$ vanishes on $G\setminus H$.
($\chi_H$ is the restriction of $\chi$ to $H$ and $[.,.]$ is the usul inner product on class functions.)
Proof. Clear from $$ \lvert H \rvert [\chi_H, \chi_H ] = \sum_{h\in H} \lvert \chi(h) \rvert^2 \leq \sum_{g\in G} \lvert \chi(g) \rvert^2 = \lvert G \rvert [\chi,\chi]. $$
Fact 1a. Let $\chi \in \Irr(G)$ and $A\leq G$ an abelian subgroup. Then $\chi(1) \leq \lvert G : A \rvert$.
Proof. For $A$ abelian, we have $$ [\chi_A, \chi_A] = \sum_{\lambda} [\chi_A, \lambda]^2 \geq \sum_{\lambda} [\chi_A, \lambda] = \chi(1).$$ Now use Fact 1.
Fact 2. Let $\chi\in \Irr(G)$ and $Z=Z(G)$, the center of $G$. Then $\chi_Z = \chi(1) \lambda$ for some linear character of $Z$, and $\chi(gz)=\chi(g)\lambda(z)$ for all $g\in G$, $z\in Z$.
Proof. The irreducible representation affording $\chi$ acts by scalars.
Fact 3. Let $\chi \in \Irr(G)$ be faithful and suppose that $g$, $h\in G$ are such that $1 \neq [g,h] (:=g^{-1}h^{-1}gh) \in Z(G)$. Then $\chi(g)=\chi(h)=0$.
Proof. $\chi(g)=\chi(g^h) = \chi(g[g,h]) = \chi(g)\lambda([g,h])$, with $\lambda$ as in Fact 2. Since $\chi$ is faithful, we have $\lambda([g,z])\neq 1$, and thus $\chi(g)=0$. Since $[g,h]^{-1} = [h,g]$, also $\chi(h)=0$.
Answer to question:
Suppose that $G$ is a $p$-group with $\lvert G \rvert \leq p^4$ and $\chi\in \Irr G$ is faithful. (We can always first factor out the kernel of $\chi$.) Let $Z=Z(G)$ and $X/Z = Z(G/Z)$, so $X$ is the second term in the ascending central series. When $Z=G$, then $G$ is abelian, so $\chi(1)=1$. Otherwise, for every $x\in X\setminus Z$, there is $g\in G$ with $1 \neq [x,g]$. But also $[x,g]\in Z$, and so $\chi(x)=0$ (Fact 3). Thus $\chi$ vanishes on $X\setminus Z$. From Fact 1 applied to $Z\leq X$ it follows
$$ \chi(1)^2 = [\chi_Z,\chi_Z] = \lvert X : Z\rvert [\chi_X, \chi_X].$$
When $X=G$, this yields $\chi(1)^2 = \lvert G : Z \rvert$, and we are finished.
Otherwise, we see at least that $p$ divides $\chi(1)$. In this case, we must have $\lvert G \rvert = p^4$, $\lvert X \rvert = p^2$. Then there is an abelian $C$ with $X < C < G$. (As in Derek Holt's answer, or observe that for any $x\in X\setminus Z$, the centralizer $C=C_G(x)$ must be a proper subgroup of order $p^3$, because $[x,G]\subseteq Z$. Because $\langle x, Z\rangle \subseteq Z(C)$, this $C$ must be abelian.) Then $\chi(1) \leq p$ by Fact 1a.