cron job to run under conda virtual environment

Posted a working solution (on Ubuntu 18.04) with detailed reasoning on SO.

The short form is:

1. Copy snippet appended by Anaconda in ~/.bashrc (at the end of the file) to a separate file ~/.bashrc_conda

As of Anaconda 2020.02 installation, the snippet reads as follows:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/USERNAME/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/home/USERNAME/anaconda3/etc/profile.d/conda.sh" ]; then
        . "/home/USERNAME/anaconda3/etc/profile.d/conda.sh"
    else
        export PATH="/home/USERNAME/anaconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

Make sure that:

• The path /home/USERNAME/anaconda3/ is correct.
• The user running the cronjob has read permissions for ~/.bashrc_conda (and no other user can write to this file).

2. In crontab -e add lines to run cronjobs on bash and to source ~/.bashrc_conda

Run crontab -e and insert the following before the cronjob:

SHELL=/bin/bash
BASH_ENV=~/.bashrc_conda

3. In crontab -e include at beginning of the cronjob conda activate my_env; as in example

Example of entry for a script that would execute at noon 12:30 each day on the Python interpreter within the conda environment:

30 12 * * * conda activate my_env; python /path/to/script.py; conda deactivate

And that's it.

You may want to check from time to time that the snippet in ~/.bashrc_conda is up to date in case conda updates its snippet in ~/.bashrc.

Found answer on stack over flow:

https://stackoverflow.com/questions/3287038/cron-and-virtualenv

The solution is to reference the python executable within the virtual environment itself. In my case, I changed the bash file to run this executable:

/home/etlservice/anaconda3/envs/sql_server_etl/bin/python

This is a solution that has worked well for me.

source /root/miniconda3/etc/profile.d/conda.sh && \
conda activate <your_env> && \
python <your_application> &

I am using miniconda with Conda version 4.7.12 on a Ubuntu 18.04.3 LTS.

I am able to place the above inside a script and run it via crontab as well without any trouble.