Unbalanced dollar signs

First example:

\newcommand\weird{$$xyz$}
\weird$

• \weird is expanded to $$xyz$ and processed;
• The first $ is identified and TeX searches for a following $ since there's a difference in how $...$ and $$...$$ are set;
• TeX find a subsequent $, thereby initiating a display math equation;
• After processing the equation interior, it searches for a closing $$;
• TeX finds $ (part of \weird) and, like before, searches for a subsequent $;
• TeX finds the closing $ (outside of \weird) and closes off the display math equation.

Using a more LaTeX-like coding, the interpretation of the expansion resembles

\[xyz\]

---

Second example:

\newcommand\curious{$uvw$$}
$\curious

• TeX find $ and looks for a following $ since there's a difference in how $...$ and $$...$$ are set;
• There is no subsequent $ (actually, the next token is \curious), so TeX opens a regular inline math expression;
• TeX expands \curious to $uvw$$;
• TeX find $ (inside \curious) and closes the inline math expression;
• TeX processes uvw;
• TeX find $ and looks for a following $ since there's a difference in how $...$ and $$...$$ are set;
• TeX finds a subsequent $ and opens a display math expression;
• No closing $$ is found - error.

Using a more LaTeX-like coding, the interpretation of the expansion resembles

\(\)xyz\[

As explained in chapter 24 of the TeXbook, a (category code 3) $ cannot be in vertical mode: if it is found when TeX is in vertical mode, horizontal mode is started and $ is reexamined.

Chapter 25 specifies

• $. A “math shift” character causes TeX to enter math mode or display math mode in the following way: TeX looks at the following token without expanding it. If that token is a $ and if TeX is currently in unrestricted horizontal mode, then TeX breaks the current paragraph […]. Otherwise TeX puts the looked-at token back into the input, enters a new level of grouping, inserts the \everymath tokens, and processes ‘⟨math mode material⟩$[…].’

The clause without expanding it is the key to understanding what you found out.

With \weird$, TeX expands \weird and so it sees $$weird$$.

With $\curious, the token following $ is not $, so TeX starts inline math mode and only after that (and after having inserted the \everymath tokens) it expands \curious, which leaves $curious$$ in the input stream. The first $ ends the just started math mode and $$ will start display math mode.

However, neither \hbox{\weird$} nor \hbox{$\curious$} would raise errors, but in this case xyz and uvw would be in text mode. Why is that? The “if TeX is currently in unrestricted horizontal mode” clause enters action: inside \hbox, TeX is in restricted horizontal mode: the rules stated above imply that in restricted horizontal mode $$ is just an empty math formula.