Cokernel within the Category of Groups

So, first off, $\pi$ is going to be the obvious projection $g \mapsto gN$.

Now, we just need to check initiality. Let $\alpha: G' \to L$ be such that $\alpha \circ \varphi = 0$. Then we need a map $\beta: G'/N \to L$ such that $\alpha = \beta \circ \pi$. There's an obvious choice for this: define $\beta(gN) = \alpha(g)$. We need to check that it's well-defined: if $gN = hN$, then $gh^{-1}\in N$.

Suppose that $\alpha(g)\neq \alpha(h)$. then $gh^{-1} \not\in \ker(\alpha)$. But $\ker\alpha$ is a normal subgroup of $G'$, and since $\alpha\circ\varphi = 0$, $\ker\alpha\supseteq\mathrm{im}\varphi$, so $\ker\alpha\supseteq N\ni gh^{-1}$, a contradiction. Thus, $\alpha(g) = \alpha(h)$, and $\beta$ is well-defined.

Finally, uniqueness of $\beta$ is clear, so we're done.

Essentially, the answer to your question is "yes".

For your side-question, it means that for all such $\alpha$, there are maps $\delta: G' \to G'/N$ and $\varepsilon: G'/N \to L$ such that $\alpha = \varepsilon\delta$.