How do you reload all vim windows at once?

The :windo command does for windows what :bufdo does for buffers. That is:

 :windo e

should cycle through all visible windows (i.e, not windows on other tabs, if any) and execute the :e command. Likewise:

 :bufdo e

would cycle through all buffers in the buffer list (i.e., no "hidden" buffers) and execute the same command.

Note that you may have buffers in the buffer list that are not currently displayed in any window. So whether to use :windo e or :bufdo e depends on what you want.

Relevant help is here: http://vimdoc.sourceforge.net/htmldoc/windows.html#list-repeat

Vim will automatically reload buffers that have been changed externally (and don't have unsaved changes) if you set the 'autoread' option.