How can I label terms in an equation using straight lines?

Manipulating the <rule thickness> and <bracket height> in

\underbracket[<rule thickness>][<bracket height>]{<arg>}

suffices. Note that I removed \mathbb{...}, simplified \mathrm{KL}, and changed \;\|\; to \mathrel{\Vert}. You may also want to use \mid instead of | in \ln p(\mathcal{D}|\theta).

\documentclass{article}

%\usepackage{amsmath}
\usepackage{mathtools}
%\usepackage{amsfonts}% not needed since I removed `\mathbb'

\newcommand{\E}[3][]{\operatorname{E}_{#2}#1[#3#1]}
\newcommand{\KL}[2]{D_{\mathrm{KL}}[#1\mathrel{\Vert}#2]}% Put `\mathrm{KL}' in braces. Thanks, @egreg!

\begin{document}

\begin{align}
\underbrace{\E{q_\phi(\theta)}{\ln p(\mathcal{D}|\theta)}}_\text{expected likelihood}
-\underbrace{\KL{q_\phi(\theta)}{p(\theta)}}_\text{complexity} \\
\underbracket{\E{q_\phi(\theta)}{\ln p(\mathcal{D}|\theta)}}_\text{expected likelihood}
-\underbracket{\KL{q_\phi(\theta)}{p(\theta)}}_\text{complexity} \\
\underbracket[0.27778ex][0pt]{\E{q_\phi(\theta)}{\ln p(\mathcal{D}|\theta)}}_\text{expected likelihood}
-\underbracket[0.27778ex][0pt]{\KL{q_\phi(\theta)}{p(\theta)}}_\text{complexity}
\end{align}

\end{document}

underline

I agree with @StevenB.Segletes: This looks rather confusing. Are you dividing the expectation by “expected likelihood”, and dividing the information/entropy by “complexity”? Even with the thick rules, the expressions in Eq. (3) appear to be (poorly typed) fractions at first glance.


You can realize it with other methods than \underbracket:

\documentclass{article}
\usepackage{mathtools}
\usepackage{amssymb}

\newcommand{\E}[3][]{\operatorname{\mathbb{E}}_{#2}#1[#3#1]}
\newcommand{\KL}[2]{D_{\mathrm{KL}}[#1\dmid#2]}
\newcommand{\dmid}{\parallel} % a semantic alias

\newcommand{\explain}[2]{{%
  \vtop{\offinterlineskip\halign{\hfil##\hfil\cr
    $\displaystyle{#1}$\cr
    \noalign{\vskip1pt\hrule height 1pt\vskip1pt}
    {\scriptsize\strut#2}\cr
  }}%
}}

\begin{document}

\begin{equation}
\explain{\E{q_\phi(\theta)}{\ln p(\mathcal{D}\mid\theta)}}
        {expected likelihood}
-\explain{\KL{q_\phi(\theta)}{p(\theta)}\vphantom{\E{q_\phi(\theta)}{}}}
         {complexity}
\end{equation}

\end{document}

I've fixed the definition of \E to have \operatorname and \mathbb switched. There is no need to do \mathrm{\operatorname{KL}} and an additional pair of braces is recommended: D_{\mathrm{KL}} is correct.

A \vphantom will make the underlines align. The | should be \mid.

enter image description here