Cannot create a crontab job for my scrapy program

This is the reason of almost all the cases where cron doesn't seem to run.

Cron always runs with a mostly empty environment. HOME, LOGNAME, and SHELL are set; and a very limited PATH. It is therefore advisable to use complete paths to executables, and export any variables you need in your script when using cron.

Also, you can use the environment variables you use on your shell.

Note that you can't use variable substitution as in shell, so a declaration like PATH=/usr/local/bin:$PATH is interpreted literally.