Does the categorical notion of a coequalizer coincide with the usual quotient objects considered in algebra?

Yes. In a fairly general class of categories, we can define an internal equivalence relation or congruence on an object $X$ to be a subobject $R \subseteq X \times X$ satisfying the axioms of an equivalence relation, suitably generalized so that they can be stated categorically (see the link for details). A congruence, being a subobject of $X \times X$, is equipped with a pair of maps to $X$, and the quotient by a congruence is the coequalizer of this pair of maps.

For groups you can check that every congruence on a group $G$ has the form

$$\{ (g, h) \in G \times G : g^{-1} h \in N \}$$

for a unique normal subgroup $N$. If we write the congruence as an equivalence relation $\sim$ on $G$, then $N = \{ n \in G : n \sim e \}$. So talking about congruences is equivalent to talking about normal subgroups, and furthermore the quotient by the congruence (the coequalizer above) associated to a normal subgroup is the quotient in the usual sense.

The point of writing things this way instead of in terms of cokernels is that this recipe generalizes to settings other than groups. For example, for rings you can check that every congruence on a ring $R$ has the form

$$\{ (r, s) \in R \times R : r - s \in I \}$$

for a unique two-sided ideal $I$, and again quotienting by this ideal corresponds to taking the coequalizer. For more general objects such as monoids and semirings it's not possible to do this kind of reduction and we really just need to work with congruences.

Given two algebraic objects like you describe (group and normal subgroup, ring and ideal, and so on), their quotient is the coequalizer of the inclusion map and the trivial (zero) map.

For instance, given a group $G$ and a normal subgroup $N$, there is an incision map and a trivial map $N\to G$. Their coequalizer is $G/N$.

This special case of a coequalizer (between the zero map and another, not necessarily injective / mono map) is called a cokernel.