How to avoid the enlarging of a \vbox by \vbox\unvbox?

After \vsplit\mybox to 45pt the box is forced to the total height 45pt:

\showboxdepth=\maxdimen
\showboxbreadth=\maxdimen
\showbox\mybox

The .log file shows:

> \box26=
\vbox(45.0+0.0)x345.0, glue set - 0.32408
.\hbox(6.94444+0.0)x345.0, glue set 306.8055fil
..\hbox(0.0+0.0)x15.0
..\OT1/cmr/m/n/10 B
..\OT1/cmr/m/n/10 l
..\OT1/cmr/m/n/10 a
..\glue 3.33333 plus 1.66666 minus 1.11111
..\OT1/cmr/m/n/10 1
..\penalty 10000
..\glue(\parfillskip) 0.0 plus 1.0fil
..\glue(\rightskip) 0.0
.\penalty -51
.\glue 8.0 plus 3.0 minus 4.0
.\glue -8.0 plus -3.0 minus -4.0
.\glue 4.0 plus 1.0 minus 3.0
.\glue(\parskip) 4.0 plus 2.0 minus 1.0
.\glue(\baselineskip) 5.05556
[...]

The outmost box is not set with its natural height, its glue components (the shrinkable parts) is reduced by factor - 0.32408. The sum of the shrinkable parts (after minus in the \glue lines of the .log file) is 6pt. Multiplied with 0.32408 the result is 1.9248. Thats the difference to the second case (modulo rounding issues).

In the latter case, after \vbox{\unvbox\mybox}, the box contents is the same, the difference is the glue setting in the topmost \vbox:

> \box26=
\vbox(46.94444+0.0)x345.0

Here the box is set with its natural width that is slightly larger than 45pt of the forced first case. The plus and minus parts of the \glue components are not needed.

The height can be forced by using \vbox to:

\setbox\mybox=\vbox to 45pt{\unvbox\mybox}

But I do not see the point for this exercise. What do you want to achieve? \vbox + \unvbox does not remove "obsolete vertical space". And in your case there is no vertical space at the beginning and end that have an effect. The vertical box starts and ends with an \hbox.

Second example

Also the second example does not remove any white space elements (glue, kern). The effect of reduced white space at the beginning is also the effect of a different glue setting. The box is forced to a box with height 90pt. The natural height of the box is 18.39996pt. Therefore the stretchable components are used and enlarged by the huge factor 6.77441.

At the end of a box the white space can sometimes be removed by \unskip and \unkern. But this will not work, if there is something invisible afterwards like \special or \write. And this trick will not work for the first part of a vertical box. Only LuaTeX allows the full inspection and modification of boxes.

If a vertical box contains glue, then its height can be set differently to its natural height. However this works both ways, the box can be made smaller or larger in height. Thus the reverse process, setting the box again in its natural height makes the box smaller or larger.

I am thankful to all given answers and explanations. They lighten what really happens. Now, I try to answer myself - hopefully, not too silly.

In the following code, the \vbox+\unvbox operation is replaced by a macro \rebox. It resets the glue with \vbox+\unvbox but forces the height of the resulting box not to grow beyond a limit (taken from the splitting).

Now, example 1 stays at 45.0pt and example 2 is allowed to reduce the visible space above.

\documentclass{article}
\usepackage{amsmath}

\def\rebox#1#2{%
  \setbox#2=\vbox{\unvbox#2}%
  \ifdim\dimexpr\ht#2+\dp#2>#1%
    \setbox#2=\vbox to \the\dimexpr#1-\dp#2\relax{\unvbox#2}%
  \fi%
}

\def\testmybox#1{%
  \par%
  {\fboxsep0pt\fbox{\copy#1}}\par%
  \edef\myboxheight{\the\dimexpr\ht#1+\dp#1\relax}%
  Total height: \myboxheight%
}

\begin{document}

\newbox\mybox
\setbox\mybox=\vbox%
{%
  Bla 1
  \begin{itemize}
  \item bla
  \item bla
  \end{itemize}
  Bla 2.
  \begin{itemize}
  \item bla
  \item bla
  \end{itemize}
}

\splittopskip=0pt%
\splitmaxdepth=0pt%
\setbox\mybox=\vsplit\mybox to 45pt%

\textbf{Example 1:}

Before \verb+\rebox+ :

{\fboxsep0pt\fbox{\copy\mybox}}\par
\edef\myboxheight{\the\dimexpr\ht\mybox+\dp\mybox\relax}%
Total height: \myboxheight

\bigskip

After \verb+\rebox+ :

\rebox{45pt}{\mybox}
\testmybox{\mybox}

\setbox\mybox=\vbox%
{%
\begin{enumerate}
\item
$\!
\begin{aligned}[t]
-2x &>4\\
x   &< \frac{4}{-2} \\
x       &< -2
\end{aligned}
$
\item
$\!
\begin{aligned}[t]
7x -1   &< 13\\
7x      &< 13 +1\\
7x      &< 14\\
x           &< \frac{14}{7}\\
x       &< 2
\end{aligned}
$
\end{enumerate}}

\bigskip

\splittopskip=0pt%
\splitmaxdepth=0pt%
\setbox\mybox=\vsplit\mybox to 90pt

\textbf{Example 2:}

Before \verb+\rebox+ :

{\fboxsep0pt\fbox{\copy\mybox}}\par
\edef\myboxheight{\the\dimexpr\ht\mybox+\dp\mybox\relax}%
Total height: \myboxheight

\bigskip

After \verb+\rebox+ :

\rebox{90pt}{\mybox}
\testmybox{\mybox}

\end{document}