How to visualize the underfull box in final output PDF files

Well, I've found a way to show the undefull boxes in the pdf. Nothing is impossible with luatex!

The process is a bit convoluted, but it works (at least in my minimal examples). But let see first the results. This is a little example document I prepared which produces a couple of Underfull vboxes.

The example

\documentclass{article}
\usepackage{fontspec}
\usepackage{lipsum}
\usepackage{nopageno}

\parskip=3mm plus 1mm
\begin{document}
1--- \lipsum[11]

\noindent\vbox to 6cm{% This should cause underfull vbox
2--- \lipsum[48]\par
3--- \lipsum[66]
}

4-- \lipsum[75]

\begin{center}
    \begin{tabular}{|c|c|}
    \hline
      \vbox to 1cm{% Another underfull vbox
        \vskip 1mm
        \hbox{Foo}
        \vskip 2mm plus 1mm
        \hbox{Bar}
      } 
                & Foo bar \\
    \hline
      Foobar    & \vbox{\hbox{Foo}\hbox{bar}}\\ % No underfull this time
    \hline
    \end{tabular}
\end{center}

5-- \lipsum[101]
\end{document}

Paragraphs marked with 2 and 3 are typeset in a \vbox too large, and the same for the first \vbox inside the table. lualatex complains:

$ grep Underfull example.log
Underfull \vbox (badness 5802) detected at line 13
Underfull \vbox (badness 1019) detected at line 25

And the result pdf shows how paragraphs 2 and 3 are too separated, because TeX had to stretch the \parskip between them to make them fill the \vbox.

The desired result

Now, I add a single line in the preamble of the above tex source:

\directlua{dofile("DetectUnderfull.lua")}

After compiling again with lualatex, the new pdf shows the underfull vboxes in red:

How was it done

My first idea was to process from lua the log file generated by tex, and search for the "Underful vbox" messages, take note of the line number in which they happened and the hook into the process_input_buffer callback to insert at that line some kind of mark. I had some problems trying to implement this approach, so I asked for help.

In the end, I got convinced that this approach was unfeasible because LuaTeX does not provide callbacks to get the messages being written in the log, and in addition file.open and file.read cannot be safely used from lua to read toe log file because that file is open and TeX is writting in it at same time.

So I tried a different approach. I hooked into vpack_filter callback, which is called when TeX has material to be packed in a vertical box and it is about to start packing it. From lua I have the opportunity of examining that box (in fact the node list composing it), throw it, let it pass, or modify it by inserting, deleting or changing nodes, all before TeX start the actual packing.

So, from this function I call node.vpack which "emulates" what TeX is about to do, and returns the result of the packing and the badness value. So I can read this badness and decide if it is a "underfull vbox" (badness > 100). In this case I modify that list by inserting nodes which "push" the red color at the beginning and "pop" it at the end. I learned from the source code of chickenize package some nifty tricks to do so.

The code

This is the content of DetectUnderfull.lua file:

-- Code to change color borrowed and adapted from chickenize package
WHAT = node.id("whatsit")
COL = node.subtype("pdf_colorstack")
color_push = node.new(WHAT,COL)
color_pop = node.new(WHAT,COL)
color_push.stack = 0
color_pop.stack = 0
color_push.cmd = 1     -- replace cmd with command if using LuaTeX >= 0.76.0 
color_pop.cmd = 2      -- replace cmd with command if using LuaTeX >= 0.76.0 


function vertical_pack(h, grcode, tam, tipo, maxd)
  local g, b
  g,b = node.vpack(h, tam, tipo)
  if (b>100) then
    color_push.data="1 0 0 rg"
    h = node.insert_before(h,h,node.copy(color_push))
    h = node.insert_after(h,node.tail(h),node.copy(color_pop))
    return h
  end
  return true
end

-- Install my filter into luatex callbacks
luatexbase.add_to_callback('vpack_filter', vertical_pack, 'vpack_filter')

Update: Thanks to uli for detecting a problem with LuaTeX 0.76.0, and to Paul Gessler for fixing it.

No. The messages in the log file are all you can use.