Inconsistent size of parentheses in Latin Modern and Computer Modern

For historical reasons a fixed size is used for the scalable delimiters, even for cm that's not clearly a good idea (see exscale package), but here it's not good at all, if you use the font at its natural scaled size then....

\documentclass[12pt]{article}

\DeclareFontFamily{OMX}{lmex}{}
\DeclareFontShape{OMX}{lmex}{m}{n}{%
   <->lmex10%
   }{}
\usepackage{lmodern}
\usepackage{amsmath}

\begin{document}



\show\big
\showthe\baselineskip
\setbox0\hbox{$\big($}\showthe\dimexpr\ht0+\dp0
\showoutput
\showbox0

An equivalence class~$[a]_{\sim} \in A / {\sim}$ consists of all the elements
in $A$ that are mapped to $b = f(a)$. By the axiom of choice, there exists a
choice function~$c \colon A / {\sim} \to A$ which selects a representative
element of each equivalence class. There exists a
function~$h \colon B \to A / {\sim}$, so that
$h(b) = h\bigl(f(a)\bigr) = [a]_{\sim}$. This allows us to construct the
function~$g = c \circ h$, which is in fact a right-inverse of $f$.
\[
    \Biggl(\biggl(\Bigl(\bigl((X)\bigr)\Bigr)\biggr)\Biggr)
\]
\end{document}

Now the size is shown as

> 14.5pt.
l.17 \showthe\baselineskip

? 

> 14.40013pt.

with \big just less than \baselineskip

---

You can redefine the \big size keeping it within \baselineskip and so avoid \lineskip glue being used within the paragraph.

\documentclass[12pt]{article}

\usepackage{lmodern}
\usepackage{amsmath}

\begin{document}


\makeatletter
\renewcommand{\big}{\bBigg@{0.92}}
\makeatother
\show\big
\showthe\baselineskip
\setbox0\hbox{$\big($}\showthe\dimexpr\ht0+\dp0
\showoutput

An equivalence class~$[a]_{\sim} \in A / {\sim}$ consists of all the elements
in $A$ that are mapped to $b = f(a)$. By the axiom of choice, there exists a
choice function~$c \colon A / {\sim} \to A$ which selects a representative
element of each equivalence class. There exists a
function~$h \colon B \to A / {\sim}$, so that
$h(b) = h\bigl(f(a)\bigr) = [a]_{\sim}$. This allows us to construct the
function~$g = c \circ h$, which is in fact a right-inverse of $f$.
\[
    \Biggl(\biggl(\Bigl(\bigl((X)\bigr)\Bigr)\biggr)\Biggr)
\]
\end{document}

The \show at the top of the document show

> 14.5pt.
l.13 \showthe\baselineskip

? 

> 13.24792pt.
<to be read again> 

showing that \big( is less than the 14.5pt baselinekip.

The .92 chosen empirically: .93 makes the font jump to the next available size, which is too big here.

You can solve your issue by forcing use of the cmex fonts. I will use the exscale like set-up from amsfonts.

\documentclass[12pt]{article}

\usepackage{lmodern}
\usepackage{amsmath}


\DeclareFontFamily{OMX}{lmex}{}
\DeclareFontShape{OMX}{lmex}{m}{n}{%
       <-7.5>cmex7%
    <7.5-8.5>cmex8%
    <8.5-9.5>cmex9%
    <9.5->cmex10%
}{}%

\begin{document}

% \showoutput

An equivalence class~$[a]_{\sim} \in A / {\sim}$ consists of all the elements
in $A$ that are mapped to $b = f(a)$. By the axiom of choice, there exists a
choice function~$c \colon A / {\sim} \to A$ which selects a representative
element of each equivalence class. There exists a
function~$h \colon B \to A / {\sim}$, so that
$h(b) = h\bigl({f}(a)\bigr) = [a]_{\sim}$. This allows us to construct the
function~$g = c \circ h$, which is in fact a right-inverse of $f$.
\[
    \Biggl(\biggl(\Bigl(\bigl((X)\bigr)\Bigr)\biggr)\Biggr)
\]
\end{document}