Running subprocess within different virtualenv with python

I think virtualenv documentation explains it nicely.

TL;DR

Runnig python venv binary directly is NOT the same as activating venv. You also have to change PATH and VIRTUAL_ENV variables accordingly (look at os.environ)

Source

$ source /path/to/ENV/bin/activate

This will change your $PATH so its first entry is the virtualenv’s bin/ directory. (You have to use source because it changes your shell environment in-place.) This is all it does; it’s purely a convenience.

If you directly run a script or the python interpreter from the virtualenv’s bin/ directory (e.g. path/to/ENV/bin/pip or /path/to/ENV/bin/python-script.py) then sys.path will automatically be set to use the Python libraries associated with the virtualenv. But, unlike the activation scripts, the environment variables PATH and VIRTUAL_ENV will not be modified. This means that if your Python script uses e.g. subprocess to run another Python script (e.g. via a !/usr/bin/env python shebang line) the second script may not be executed with the same Python binary as the first nor have the same libraries available to it. To avoid this happening your first script will need to modify the environment variables in the same manner as the activation scripts, before the second script is executed.


The accepted answer does not address the problem of 'activating' a virtualenv in a subprocess.

If you start your application with a call to the python executable, like in your example it is actually very simple: you only have to explicitly point to the executable in the virtualenv.

import subprocess

subprocess.Popen(["virtualenv1/bin/python", "my_script.py"])
subprocess.Popen(["virtualenv2/bin/python", "my_other_script.py"])

will start the processes in the respective virtualenvs.

Important

To address the concerns voiced in the comments:

If you want to run a subprocess and be sure to use the same interpreter that the current process is running in you have to use sys.executable. Also available: sys.exec_prefix to access the site-specific directory prefix where the platform-dependent Python files are installed.

If you want a much more in depth discussion of this topic, have a look at this pull request.