Airflow DAG Running Every Second Rather Than Every Minute
Try adding catchup=False
in the DAG()
. It might be that your DAG is trying to backfill because of the start_date
that you have declared.
Your schedule_interval
on the DAG is correct: */1 * * * *
is every minute.
You can also remove start_date
and schedule_interval
from default_args
since they're redundant with the kwargs provided to the DAG.
If you changed the schedule from when you first created this DAG, it's possible Airflow's gotten confused. Try deleting the DAG in the database, and then restarting the scheduler and webserver. If you are on the master branch of Airflow, it's as simple as $ airflow delete_dag my_dag
; otherwise, the linked answer explains how to do it on other versions.
I boiled your code down to this to check and it is definitely running one DAG run per minute when run inside the master branch of Airflow.
from datetime import datetime
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
default_args = {
'owner': 'airflow',
'depends_on_past': False,
}
dag = DAG(
dag_id='airflow_slack_example',
start_date=datetime(2018, 6, 4),
schedule_interval='*/1 * * * *',
default_args=default_args,
)
test = BashOperator(
task_id='test',
bash_command='echo "one minute test"',
dag=dag,
)
DAG runs: