For $G$ group and $H$ subgroup of finite index, prove that $N \subset H$ normal subgroup of $G$ of finite index exists

Your definition of $\varphi$ looks fine. Anything in the kernel must in particular fix $H$, and $gH = H$ is equivalent to $g \in H$. On the other hand I think $N = \ker \varphi$ can be a proper subgroup of $H$. As an example, which is silly because the group is finite, if you take $G = S_3$ and $H = \{1, (12)\}$ then this process produces $N = \{1\}$.

For the second question, this is just the "first" isomorphism theorem.