Difference between ' and " on command line (bash)?

Take a look at the bash man page. There's an entire section on quoting. Because this licensed under the GFDL, which is not compatible with the CC-BY-SA license used here, I won't quote the whole thing, but really reading that is the most definitive answer.

In summary, single quotes stop all interpretation -- the string is rendered literally. Double quotes leave $ (dollar sign), ` (backquote) as special, and \ (backslash) as special when followed by certain other characters. And ! will be treated specially if the history expansion feature is enabled (which it is by default).

In practical use, the $ is the big deal, as one often may want the various expansions it enables to (variables and more), while still preventing the shell from muddling most of the command line.