DAG not visible in Web-UI
I have the same issue. To resolve I need to run scheduler
airflow scheduler
Without this command, I don't see my new DAGs BTW: the UI show me warning related to that problem:
The scheduler does not appear to be running. Last heartbeat was received 9 seconds ago. The DAGs list may not update, and new tasks will not be scheduled.
We need to clarify several things:
- By no means you need to run the DAG file yourself (unless you're testing it for syntax errors). This is the job of Scheduler/Executor.
- For DAG file to be visible by Scheduler (and consequently, Webserver), you need to add it to
dags_folder
(specified inairflow.cfg
. By default it's$AIRFLOW_HOME/dags
subfolder).
Airflow Scheduler checks dags_folder
for new DAG files every 5 minutes by default (governed by dag_dir_list_interval
in airflow.cfg
). So if you just added a new file, you have two options:
- Restart Scheduler
- Wait until current Scheduler process picks up new DAGs.
Run airflow list_dags
to check, whether the dag file is located correctly.
For some reason, I didn't see my dag in the browser UI before I executed this. Must be issue with browser cache or something.
If that doesn't work, you should just restart the webserver with airflow webserver -p 8080 -D