What is dimensional consistency, mathematically?

Buckingham Pi Theorem. See e.g. chapter 1 of Bluman and Anco, Symmetry and Integration Methods for Differential Equations.

You can check dimensional consistency all along. $\sin x$ or $\exp x$ are only defined if $x$ is dimensionless. If you take the sine or exp of some combination, it has to be dimensionless. This is a very effective way to catch errors in a derivation. Even if you have a dimensionless problem, it has to stay correct if you attach dimensions. If you have the standard quadratic, $ax^2+bx+c=0$ you can just assign $x$ dimensions of length. Then $a$ must be length$^{-2}$ and so on. If your final result doesn't respect this, you have made an error.