$H$ is normal subgroup of $G$ iff $gHg^{-1}=H$ for every $g$ of $G$

I think a more elegant proof is possible.

Clearly $gHg^{-1}=H\iff gHg^{-1}g=Hg\iff gHe=Hg\iff gH=Hg, \forall g\in G$, simply by multiplying by $g$ on the right (and simplifying $g^{-1}g=e$ and $He=H$).

You have proved that $Hg\subseteq gH$. One might say "by symmetry" the other direction holds, or just prove it directly, using basically the same argument:

Let $m\in gH$ be arbitrary. Then $m=gh$ for some $h\in H$. Now $mg^{-1}=ghg^{-1}=h'$, for some $h'\in H$. Hence $m=h'g$, so $m\in Hg$. This proves that $gH\subseteq Hg$.