run selenium with crontab (python)
on MacOS Catalina only this command worked for me
* * * * * export DISPLAY=:0 && export PATH=$PATH:/usr/local/bin && /usr/bin/python3 ~/Documents/Scripts/my_script.py
The most evident problem with trying to launch a browser from cron
is that even if you have X running on your machine, the DISPLAY
environment variable is not set for processes running from your crontab
so launching a browser from there will fail.
Solutions range from the trivial to the super sophisticated. A trivial solution would be to accept that your script won't run if there is no X running and manually set DISPLAY
to :0
, which is the default display number for the default X server that Ubuntu starts.
For instance, if I put this command in the command
column of a crontab
line, Chrome starts without issue:
DISPLAY=:0 google-chrome
The complete line in the a user-specific crontab file would be something like:
0 * * * * DISPLAY=:0 google-chrome
If you want to run a python script that starts chrome through selenium, the line would instead look like:
0 * * * * DISPLAY=:0 python my_script.py
The command string is just sent as-is to the shell so in the last example the string DISPLAY=:0 python my_script.py
would be just passed to the shell. It is common shell syntax to interpret a variable assignment given immediately at the start of the command as setting an environment variable. (It is certainly the case for dash
and bash
, one of which is likely to be the default shell in most installations.) So the command that the shell interprets sets the environment variable DISPLAY
to the value :0
and then runs python my_script.py
. Since python
inherits its environment from the shell that started it, the variable DISPLAY
is :0
for it too.
Setting DISPLAY=:0
like I show above sets the variable only for the command that follows. It is also possible to set DISPLAY
to :0
for all commands executed by the crontab. For instance in the following user-specific crontab:
DISPLAY=:0
30 * * * * google-chrome
0 * * * * python my_script.py
the line DISPLAY=:0
sets the environment variable DISPLAY
both for the execution of google-chrome
and python my_script.py