Run multiple cron jobs where one job takes a long time

Each cron job is executed independent of any other jobs you may have specified. This means that your long-lived script will not impede other jobs from being executed at the specified time.

If any of your scripts are still executing at their next scheduled cron interval, then another, concurrent, instance of your script will be executed.

This can have unforeseen consequences depending on what your script does. I would recommend reading the Wikipedia article on File Locking, specifically the section on Lock files. A lock file is a simple mechanism to signal that a resource — in your case the someScript3.sh script — is currently 'locked' (i.e. in use) and should not be executed again until the lock file is removed.

Take a look at the answers to the following question for details of ways to implement a lock file in your script:

  • How to make sure only one instance of a bash script runs?

Not sure what you mean by appropriate time. Cron will start jobs at the time it scheduled to do so. It does not check other scheduled jobs nor other instances of a job.

So whatever valid jobs you define will be started at the time defined. Any job that runs longer than the defined interval will be started multiple times. It is the responsibility of whoever wrote the job to prevent it from actually running multiple times if that is required. By e.g. checking a lock file or PID file or something.

There are obvious limitations to the amount of processes that can run in parallel but those are not cron specific.


In addition to other answers, especially the link posted by @soulcake: If you schedule a long running command with a too short interval, cron will happily execute the second before the first one completes (unless there's some kind of mutex implemented in the command).

That often slows down the original command even further, leading to another instance being run before the previous ones complete, etc. Or it might be undesirable for other reasons.

General way to prevent is to condition running the command with a guard that ensures that a previous command is not running. For example:

10 * * * * pgrep my_slow_command >/dev/null || /usr/local/bin/my_slow_command

Make sure that pgrep matches the command's name when it runs, e.g. python scripts have python as the name of the executable, which is probably not specific enough and you would have to match against the python's script name as well.

10 * * * * pgrep -f my_script.py || /usr/local/bin/my_script.py

(pgrep without '-f' option matches bash script names, though)

If you can't use pgrep for some reason:

10 * * * * ps ax | grep [m]y_command || /usr/local/bin/my_command

The brackets are used to avoid matching the grep command itself.

Tags:

Cron