Running Firefox in an Ubuntu VM over SSH opens host machine's Firefox

When Firefox starts, it looks for a Firefox window running on the same display, and if it finds one, it focuses this window (and if you pass a URL on the command line, it opens a new tab to load the URL in the existing window).

You must have run SSH with X11 display forwarding. Since X11 forwarding is active, all GUI programs that you start in the SSH session will be displayed on the local machine.

If you X11 forwarding was not active in the SSH connection, then GUI applications run from the SSH session would have nowhere to display. They'd just complain “Error: no display specified” or some similar error message.

X11 is inherently network-transparent, so it doesn't have a notion of “the local display”. The display is whatever you tell the application is the display. There can be multiple local displays, e.g. in the case of a multiseat configuration. There isn't one “true” display like there is with Windows. If you're running a program remotely and you want it to display on the monitor of the remote machine, you need to run an X server on the remote machine and you need to explicitly tell the program to connect to that display. By default, if you do nothing, programs will be displayed on the machine that you're in front of.

You want firefox --no-remote (or firefox --new-instance). Otherwise, it will "reuse" your existing firefox process.