Every group of order 4 is isomorphic to $\mathbb{Z}_{4}$ or the Klein group

As @rain1 pointed out, we have a group $G=\{1,a,b,ab\}$, where $a$ and $b$ are different, commute and are not equal to $1$. Let us call $ab=ba=c$. Observe that $a \neq c$ and $b \neq c$. Now look at $a^2$. Then $a^2 \notin \{a,c\}$, so either $a^2=1$ or $a^2=b$. Symmetrically, either $b^2=1$ or $b^2=a$. So there are $4$ cases to consider, but by symmetry in $a$ and $b$ this boils down to only $2$. Firstly, $a^2=1$ and $b^2=1$, in this case $G \cong V_4$. And secondly, if $a^2=1$ and $b^2=a$, then $b^4=1$ and $G \cong C_4$. So no need of the structure theorem of abelian groups.

For groups of order $6$ you can proceed in a similar, but slightly more complicated way. Just applying elementary means. No Lagrange, no Cauchy.