Consequences for virtual env when system's Python is removed and/or updated
Bad news: You're using venv
, and venv
virtual environments are not completely standalone. Even when created with --copies
, all it copies is the python
executable itself, not the standard libraries, not even (if your install creates a shared libpython
rather than a static one) libpython
; it relies on the system copy. The virtual environment will break if the Python install it was based on goes away. How it breaks will differ based on how it was created. For example, if you created it with:
python3 -mvenv path/to/venv
when python3
meant Python 3.7, then replaced python3
with Python 3.8, then you can fix up the virtual env for the new version with:
python3 -mvenv --upgrade path/to/venv
but your installed third party packages will be (effectively) gone (they'll be in path/to/venv/lib/python3.7
, but Python 3.8 will only be looking in path/to/venv/lib/python3.8
), so you'll have to reinstall them.
If you created the virtual environment with:
python3.7 -mvenv path/to/venv
then it's completely broken (at least as documented), the --upgrade
switch is only documented to work for upgrading when Python was upgraded in place; since the new Python would not be named python3.7
, you can't upgrade in place. That said, --upgrade
really only works well when upgrading micro versions (from 3.7.1 to 3.7.2 or the like), due to aforementioned per-minor version lib/pythonX.Y
directories, so either way, you're best off just creating a new virtual environment from scratch.
To be clear, the third party virtualenv
package does not have this limitation if and only if the system Python installation statically linked libpython
. Oddly, while the --always-copy
flag will make it copy the main binary and the standard library modules, it doesn't cause libpython
itself (the interpreter core) to be copied, so if the main binary depends on a system copy of libpython.so
, then removing the system copy breaks the virtual environment. If you did use --always-copy
and your python
executable statically linked libpython.a
(ldd /path/to/python3
should show no libpython
dependency), then yes, virtualenv
makes much more heavyweight (in local tests on 3.6, forcing copies via the appropriate switch, freshly created venv
environments were ~11 MB, while virtualenv
environments were ~48 MB; sadly, my python
dynamically links libpython.so
, so it still wouldn't work) virtual environments that should survive the removal of the system installed copy of Python.
In any event, it's probably best to leave your Python 3.7 install in place, and just upgrade to 3.8 without removing 3.7 (are you really hard up for a few dozen MB of disk space?). Even if you replace python3
with the new 3.8 install, python3.7
, libpython3.7m.so.1.0
or the like and the rest of the 3.7 standard library will continue to exist for the virtual environment to depend on; at worst you might need to manually change the symlink in the virtual environment to point to /path/to/python3.7
rather than /path/to/python3
to keep using the old version (including all your installed third party packages).
An alternative to trying to keep the old virtual environment working is to just back up the installed state of that virtual environment, delete it, install the new Python, create a fresh virtual environment, and then use the backed up state to reinstall all your packages in the upgraded virtual env. An example might be:
$ source ~/path/to/venv/bin/activate
$ pip freeze > installed_libs.txt
$ deactivate
$ rm -rf ~/path/to/venv
$ ... install new Python/remove old Python ...
$ python3 -mvenv ~/path/to/venv
$ pip install -r installed_libs.txt # Optionally add --upgrade to install latest, not fixed versions