Order of automorphism group of cyclic group

Since an automorphism of $G$ should map a generator of $G$ to a generator of $G$ it's enough to know how many generators does $G$ have.

If $G=\{e,g,g^2,...,g^{m-1}\}$ then a $g^i$ generates G if and only if $\operatorname{gcd}(i,m)=1$.

$\lvert \operatorname{Aut}(G)\rvert=\phi(m)$ where $\phi(m)$ is Euler's function.

For a more detailed proof:

1. Let $G=\langle g\rangle$ and $f\in\operatorname{Aut}(G)$.
2. Then $f(g)=g^i$ for some $i$. If $f$ is an isomorphism $\langle g^i\rangle =G$ and this happens only if $\operatorname{gcd}(i,m)=1$.
3. On the other hand every homomorphism $f:G\rightarrow G$ with $f(g)=g^i$ is an isomorphism when $\operatorname{gcd}(i,m)=1$, so $\lvert \operatorname{Aut}(G)\rvert=\phi(m)$.

Hint:

For a given cyclic group $G,\;\text{with}\;\; |G| = m$: $$\text{Aut}\,(G) \cong \mathbb{Z}_m^*\tag{$\;^*:\;\;$multiplicative group}$$

Hence $\text|{Aut}\,(G)| = |\mathbb{Z}_m^*|.$

Since an automorphism of $G$ should map a generator of $G$ to a generator of $G$, it's enough to know how many generators $G$ must have.

So $|\text{Aut}(G)|=|\mathbb{Z}_m^*| = \phi(m)$ where $\phi(m)$ is Euler's totient function.

---

(Note: you do not need that $m = p$, a prime; it suffices to know that $G$ is cyclic and finite.)