Show that every group of prime order is cyclic
As Cam McLeman comments, Lagranges theorem is considerably simpler for groups of prime order than for general groups: it states that the group (of prime order) has no non-trivial proper subgroups.
I'll use the following
Lemma
Let $G$ be a group, $x\in G$, $a,b\in \mathbb Z$ and $a\perp b$. If $x^a = x^b$, then $x=1$.
Proof: by Bezout's lemma, some $k,\ell\in\mathbb Z$ exist, such that $ak+b\ell=1$. Then $$ x = x^{ak+b\ell} = (x^a)^k \cdot (x^b)^\ell = 1^k \cdot 1^\ell = 1 $$
(If you know a little ring theory, you might prefer to notice that the set $\{i | x^i=1\}\subseteq \mathbb Z$ forms an ideal which must contain $(a,b)=1$ if it contains $a$ and $b$.)
The question
Now let $P$ be an arbitrary group of prime order $p$. Consider any $x\in P$ such that $x\neq 1$ and consider the set $$ S = \{ 1, x, x^2 , \dots , x^{p-1} \}\subseteq P.$$ First assume two of these elements are equal, say $x^u=x^v$ and $u<v$ without loss of generality. Then $x^{v-u}=1$ and $1\leq v-u \leq p-1$. But then surely $v-u \perp p$. By the lemma, $x^{v-u} = x^p = 1$ now implies that $x=1$, a contradiction so every two members of $S$ must be different.
But then $|S|=p$. This implies $S=P$ and $P=\langle x\rangle$ is cyclic.
The answer is fairly simple once Lagrange's Theorem is quoted. We have no proper subgroups of smaller order. We only need to prove the uniqueness of the group of that size. For this note that given any element of such a group, continue to take powers of it ... This series $x^r$ has to terminate because of closure. The series also has to exhaust all the elements of the group, otherwise we will have subgroups of a smaller order.
Thus we have proven that every group of prime order is necessarily cyclic. Now every cyclic group of finite order is isomorphic to $\mathbb{Z}_n$ under modular addition, equivalently, the group of partitions of unity of order $|G|$. Thus the uniqueness is proved.
This is a proof of Cauchy's theorem that does not use Lagrange, it is due to James McKay. It has an uncanny similarity to the proof of Fermat's Little theorem using necklaces.
Let $G$ be a group of order $np$. Then there are $(np)^{p-1}$ solutions to $g_1g_2\dots g_p=1$ since for any values of $g_1,g_2,\dots ,g_{p-1}$ there is a unique inverse for $g_1g_2\dots g_{p-1}$. We call $S$ the set of solutions, we have asserted $|S|$ is a multiple of $p$.
Notice if $g_1,g_2\dots g_p=1$ then $g_ig_{i+1}\dots g_pg_1\dots g_{i-1}=1$ also.
Divide $S$ in rotation classes. Where $s$ is in the same class as $s'$ only if they are rotations of each other. Notice all classes have size $1$ or $p$ (this uses $p$ is prime).Therefore the number of classes of size $1$ is multiple of $p$. Since $\underbrace{1,1\dots ,1}_\text{p times}$ makes up a class of size $1$ there must be another, this provides the desired element of order $p$.
We may now use Cauchy Theorem to determine a group $G$ of order $p$ has an element $g$ of order $p$, this element generates a cyclic subgroup of order $p$. This subgroup must be $G$ so $G$ is cyclic.