Exclamation mark variable `$!`

$! is the PID of the most recent background command.

See this excellent answer for other special parameters.

The shell checks for new mail after a command finished and $MAILCHECK or 600 seconds have passed.

echo $! prints the PID of the last background process. After that, a check for mail may happen.

$! is replaced by the process identifier of the last process placed in the background in the current shell, if any. The bash manual provides details of all such special parameters.

To see this in action:

echo $!

should print 0 (assuming no jobs have been backgrounded)

man bash &
echo $!

should print the process identifier of the man process which is in the background. The full output should look something like

$ man bash &
[1] 4319
[1]  + 4319 suspended (tty output)  man bash
$ echo $!
4319