Why does \vspace, after section heading, sometimes jump/snap in discrete steps?

So, if anyone can explain what is happenning here - and is there a possibility for "smooth" vertical positioning of overflown paragraphs, please post back...

As Stephan remarked this is a wonderful question to look at, but I think with all the animation and the coding around it you have fairly effectively hidden the problem.

The culprit is the center environment just in front of the space that you modify. The problem is that this environment surrounds itself with flexible glue (ie with a stretch and shrink component). The exact values in your case are

10pt plus 3pt minus 5pt

So in the place where you have your space you really have

\mylength + 10pt plus 3pt minus 5pt

As it was remarked in other answers TeX attempts to find a page break with minimal badness. So, if we assume that the page break TeX found just naturally fits (ie no shrink or stretch is necessary) then you just get \mylength + 10pt at this point. Now if you add 1pt to \mylength then the last line wouldn't fit any more on the page, except that your extra point can be absorbed by the shrink on the page.

Now if the only shrink available is in this very place (which it is in your example) then your extra point will just be swallowed and the space remains the same. Same story for adding another pt and another one. (If there are several places with shrink on the page then the shrink gets distributed evening across those places, so in that cases you might see a small increase but less than your step value.)

But the moment you have added 4pt it can't be compensated by the shrink available in your example. Thus the last line now really doesn't fit any more on the page and thus TeX needs to use "stretch" to fill the missing line (minus your 4pt), i.e., 8pt or so depending on the \baselineskip setting.

So no more shrink and instead a stretch and your space makes a jump. Adding further pts will increase your space while the need for stretch gets less until you reach the point where the page again is "natural" without a need for stretch and then everything repeats.

Now the \raggedbottom setting only effects how the final page is typeset (after breaking it). Basically, the cut-off page is put into a bot of \textheight and in case of \raggedbottom a \vfill is added to the bottom (more or less). Now in the "shrink" situation this doesn't really make a difference as the page is already overfull. And in the stretch situation it means that the stretch in the end is only applied to the bottom and not to the stretchable parts in the middle of the page.

So in summary: to avoid your mystery you need to ensure that the space you are trying to adjust has no shrink or stretch component either as part of your adjustment or as part of space visually next to it. Only then you can ensure that the space actually behaves as you expect

LaTeX by default vertically aligns the page content to spread on the whole page, from the top to the bottom, with the exception before manual page break (such as new chapter or usage of \newpage or \clearpage or at the end of the document.

To achieve this, every vertical space at the page is equipped with some stretch. Even baseline skip has a stretch so the paragraph lines does not need to be at the same distance. In your case, it seems that LaTeX uses the (negative) stretch in the newline \\ you use to produce the space. Try using \vspace*{10.0pt} instead.

It is generally not a good idea to remove stretches from the format-defining lengths, because if you had no vertical stretch on the page at all (i.e. no inter-paragraph nor inter-line nor section spacing stretches), the page will become "short", LaTeX would not be able to fill the whole page and Warning: Underfull \vbox while \output is active would appear in the log file. The pages will no have equal height and this makes your document look ugly.

The GIF images are great and the bash script kung-fu even greater, however it is instructive to watch both the top as well as the bottom of the page. Here is an image of a slightly different page firstly without any \vspace added.

Note the text starts on line 4 and ends with the word 'donee'.

The next two images set \vspace* at 16pt and 18pt .

Set \vspace*{16pt}

Set at ``\vspace{18pt}`

The jumps you experiencing (especially if you use vspace rather than the starred version`, is TeX's attempt to always produce a page with a minimum of badness.

This is an old problem and in the first published work on typography (1683) by Joseph Moxon, Mechanick Exercises: Or, the Doctrine of Handy Works Applied to the Art of Printing, reference is made to adding spaces to balance the text so that the last line always falls at the same place and to even the page.

The best place to start to understand the issues involved is Michael Plass, Ph.D. Thesis, Optimal Pagination techniques for Automatic Typesetting Systems.

The MWE code for the images follows, it will be nice if you could combine it with your GIFs. It is instructive to note and observe TeX's attempt to always end up in a nice page display.

\documentclass[10pt,twoside]{article}
\usepackage{fancyhdr,lipsum}
\makeatletter
 \newsavebox{\@linebox}
 \savebox{\@linebox}[3em][t]{\parbox[t]{3em}{%
   \@tempcnta\@ne\relax
   \loop{\underline{\the\@tempcnta}}\\
     \advance\@tempcnta by \@ne\ifnum\@tempcnta<48\repeat}}
 \pagestyle{fancy}
 \fancyhead{}
 \fancyfoot{}
 \fancyhead[CO]{\scriptsize How to Count Lines}
 \fancyhead[RO,LE]{\footnotesize\thepage}
%% insert this block within a conditional
 \fancyhead[LE]{\footnotesize\thepage\begin{picture}(0,0)%
      \put(-26,-25){\usebox{\@linebox}}%
      \end{picture}}
 \fancyhead[LO]{%
    \begin{picture}(0,0)%
      \put(-18,-25){\usebox{\@linebox}}%
     \end{picture}}
\fancyfoot[C]{\scriptsize Draft copy}
%% end conditional
\makeatother
\begin{document}
%\rule{10pt}{18pt}Latey
\section*{Introduction}
\vspace*{3pt}
\lipsum[1]

$$f_{nk}=\sum\,{\frac{n!}{
1!^{k_1}\,k_1!\,2!^{k_2}\,k_2!\,3!^{k_3}\,k_3!\,\ldots}}\; 
f_1^{k_1}f_2^{k_2}f_3^{k_3}\ldots\;,$$
summed over all $k_1,k_2,k_3,\ldots\geq 0$ 

\lipsum[3]

$$f_{nk}=\sum\,{\frac{n!}{
1!^{k_1}\,k_1!\,2!^{k_2}\,k_2!\,3!^{k_3}\,k_3!\,\ldots}}\; 
f_1^{k_1}f_2^{k_2}f_3^{k_3}\ldots\;,$$
summed over all $k_1,k_2,k_3,\ldots\geq 0$ 

\lipsum[5-12]
\end{document}