How to tell Jenkins to use a particular virtualenv python

First, you should avoid using ShiningPanda because is broken. It will fail if you try to run jobs in parallel and is also not compatible with Jenkins2 pipelines.

When builds are run in parallel (concurrent) Jenkins will append @2,@3... to the workspace directory so two executions will not share the same folder. Jenkins does clone the original worksspace so do not be surprised if it will contain a virtualenv you created in a previous build.

You need to take care of the virtualenv creation yourself but you have to be very careful about how you use it:

• workspaces folder may not be cleanup and its location could change from one build to another
• virtualenvs are knows to get broken when they are moved, and jenkins moves them.
• creating files outside workspace is a really bad CI practice, avoid the temptation to use /tmp

So your only safe option is to create an unique virtual environment folder for each build inside the workspace. You can easily do this by using the $JOB_NUMBER environment variable.

This will be different even if you have jobs running in parallel. Also this will not repeat.

Downsides:

• speed: virtualenvs are not reused between builds so they are fully recreated. If you use --site-packages you may speedup the creation considerably (if the heavy packets are already installed on the system)
• space: if the workspace is not cleaned regularly, the number of virtualenvs will grow. Workaround: have a job that cleans workspaces every week or every two weeks. This is also a good practice for spotting other errors. Some people choose to clean workspace for each execution.

Shell snippet

      #/bin/bash
      set -euox pipefail

      # Get an unique venv folder to using *inside* workspace
      VENV=".venv-$BUILD_NUMBER"

      # Initialize new venv
      virtualenv "$VENV"

      # Update pip
      PS1="${PS1:-}" source "$VENV/bin/activate"

      # <YOUR CODE HERE>

The first line is implementing bash string mode, more details at http://redsymbol.net/articles/unofficial-bash-strict-mode/

You should install one of python plugins. I've used ShiningPanda. Then you'll be able to create separate virtual environment configurations in Manage Jenkins > Configure System > Python > Python installation. In job configuration there will be Python Builder step, where you can select python environment.

Just make sure you're not starting Jenkins service from within existing python virtual environment.