Looking for a simple proof that groups of order $2p$ are up to isomorphism $\mathbb{Z}_{2p}$ and $D_{p}$ .

Since we are allowed to use Sylow, we can assume $G$ is generated by $x,y$ with $x^p=y^2=1$, where $\langle x \rangle \lhd G$, so $y^{-1}xy = x^t$ for some $t$ with $1 \le t \le p-1$. Then $x = y^{-2}xy^2 = x^{t^2}$, so $p$ divides $t^2-1 = (t-1)(t+1)$, hence $p$ divides $t-1$ or $t+1$ and the only possibilities are $t=1$ or $p-1$, giving the cyclic and dihedral groups.

Since the $2$-Sylow subgroup is cyclic, the group has a normal $2$-complement (corollary to Burnside's transfer theorem), which means that the $p$-Sylow subgroup is normal (or just use that any subgroup of index $2$ is normal). Thus, the group is a semidirect product of a cyclic group of order $p$ and one of order $2$. Since the Automorphism group of the cyclic group of order $p$ has a unique subgroup of order $2$, this means that there can only be one non-trivial such semidirect product, and since $D_p$ is such a semidirect product, it must be it. If the semidirect product is trivial, we of course get the cyclic group of order $2p$.