\lstinline!Foo! vs \lstinline{Foo}

The reason they exist next to each other is that

• inside {...}, the LaTeX "special characters" are treated differently, so you cannot use $, because it is a character for math mode.
• inside !...!, you can use anything but !.

As far as what is "good habit": whatever you want. I consider {} less confusing and would use this as default, and switch to !! only when necessary. On the other hand, if most of your entries need !!, then probably make this your default.

I found a slight difference between {} and using an arbitrary symbol as delimiter:

When trying to use mathescape inside math mode, {} behaves not as expected, !! works:

\documentclass{article}

\usepackage{listings}
\usepackage{amsmath}

\begin{document}
With ``!!'': \lstinline[mathescape]!a$\beta$c! in math mode gives
\begin{align}
    \lstinline[mathescape]!a$\beta$c!
\end{align}

With ``\{\}'': \lstinline[mathescape]{a$\beta$c} in math mode gives
\begin{align}
    \lstinline[mathescape]{a$\beta$c}
\end{align}

\end{document}

gives (to the amount I can represent that here)

With “!!”: aβc in math mode gives

aβc (1)

With “{}”: aβc in math mode gives

βac (2)

and the first align block compiles without error, while the second one does not. (Improper alphabetic constant and Undefined control sequence in line 15.) So, I don't see that {} handles $$ differently, as stated in the previous answer, but braces and other symbols do obviously behave differently (and it seems that there is indeed still something wrong with the braces or possibly with amsmath-align. This would need further experimentation, reading or somebody who understands the source).

Otherwise, I mostly use them as convenient, my usual way is to use {} unless my code contains braces, in which case I use !!.