How do I increase my iterm (tmux) window scrollback (not the line scrollback)

Putting this in my .tmux.conf file was the trick:

set -g history-limit 20000

For tmux you can alter its scrollback buffer with

set-option history-limit 10000

The default is 2000.

You can put this directive in your ~/.tmux.conf or at the tmux command prompt (prefix + :).

It looks like iTerm is integrated with tmux. See: https://gitlab.com/gnachman/iterm2/wikis/TmuxIntegration