\color{.} \] \par leads to extra space
The \color{red} instruction saves something to be executed at the end of the current group. If there is no blank line after the display, this has no apparent side effect. But if there is a blank line, the saved tokens make a “rest of paragraph” before the implicit \par instruction is executed. So you get an empty line.
Solution: use \textcolor or wrap the part to be affected by \color in \begingroup\endgroup.
On the other hand, blank lines after a math display are rather the exception than the rule; a blank line before a math display is a capital sin, penitenziagite.
\documentclass{article}
\usepackage{color}
\begin{document}
Remark above display math
\[a^2+b^2<c^2\]
Remark under display math at its usual position
\[\begingroup\color{red}a^2+b^2>c^2\endgroup\]
Remark is further away with color command
\end{document}
Colors are implemented as switchers-specials (pdfliterals/colorstacks) inserted to the output. This is independent off the TeX grouping mechanism, but users expect that colors will work dependent on TeX grouping mechanism. So, macros use the TeX primitive \aftergroup whenever the color is changed by user in a group in order to restore the previous color after the group is closed. But this restoring command is another switcher-special (pdfliteral/colorstack) and it creates a node in the typesetting material. Your example is:
$$ formula color-setting $$
the color-setting puts node about color and runs \aftergroup. When the group is closed then next node about restoring color occurs. It is inserted after display mode in immediately followed horizontal mode. Then the empty line does \par. The paragraph includes the mentioned color node, it is not empty, so you can see the empty line.
If you type
$$ {formula color-setting} $$
in your syntax:
\[{ a^2+b^2>c^2\color{red} }\]
then the \aftergroup node ocurrs after the closing }. Next, the display mode is closed and \par (empty line) is processed at empty horizontal list, so no empty line is created.
Of course, LaTeX developers can define \[ as $$\begingroup and \] as \endgroup$$ (but they did not do it). But if you use classical TeX syntax $$...$$ combined by colors inside display mode then the problem is still here.
Unfortunately, there is only \aftergroup TeX primitive. More natural for color handling would be \atendgroup primitive which would be processed right before the closing group (no after it). But such primitive does not exists and the TeX source is not so clear in this issue, so adding such primitive is not simple.
LuaTeX uses another approach using its \attributes to handling color. This enables to not insert any \aftergroup color node and color specials are processed at back end as the result of attribute processing.