In bash, how do I escape an exclamation mark?

Since you do not depend on bash to expand variables in your commit message you could use single quotes instead. Strings in single quotes are not expanded by bash.

bzr commit -m 'This does work!' 

Here is another method if you want double quotes as well as the exclamation:

echo "It's broken"'!'

This works even if the ! is not at the end of the line.

For instance:

echo "hello there"'!'" and goodbye"

Bonus: A similar technique can be used to escape any text in Sh or Bash (with the help of sed): see the first option in this answer. Further, if you have bash-completion installed, you likely have the quote() function available already.

Turn off history expansion:

set +H

or

set +o histexpand

You can add one of those commands to your ~/.bashrc if you usually don't use history expansion.

Bash 4.3 added a special case:

the history expansion character is also treated as quoted if it immediately precedes the closing double quote in a double-quoted string