Unramified prime

This is known as Stickelberger's Theorem, and the proof is not exactly straightforward. Here is how it goes: by Dedekind criterion the number of primes above $p$ is exactly the number of irreducible factors of $f$ modulo $p$, because $p$ does not divide $\Delta(f)$. So write $f=g_1\ldots g_r$ in $\mathbb F_p[x]$. If $r=1$, the claim holds because on the one hand the Galois group of $g_1$ over $\mathbb F_p$ is $C_{\deg g_1}$, and on the other hand the Galois group of an irreducible polynomial of degree $n$ is contained in $A_n$ if and only if the discriminant is a square. To get the general case, just use the fact that $\Delta(f)$ is $\prod \Delta(g_i)$ up to a square.