Trigger Airflow DAG run
Description
The Trigger Airflow DAG run action starts a run of a DAG in Apache Airflow 3, through the Airflow REST API. By default it waits until the run finishes: the action succeeds when the run succeeds and fails when the run fails.
Use it to start work in Airflow from a Hop workflow, for example a DAG that should run after Hop loaded new data.
The action needs an Airflow Connection.
Options
DAG run
| Option | Description |
|---|---|
Airflow connection |
The Airflow Connection to use. |
DAG id |
The id of the DAG to run. |
DAG run id |
The id of the new run. Leave it empty to let Airflow choose one. A run id must be unique within the DAG. |
Logical date |
An ISO 8601 date and time with a time zone, for example |
Run conf
The DAG receives the run conf as dag_run.conf.
| Option | Description |
|---|---|
Run conf (JSON) |
A JSON object. Use it for numbers, lists and nested values, for example |
Key, Value |
Keys with text values. A key in this table replaces the same key in the JSON. |
Wait
| Option | Description |
|---|---|
Wait for the DAG run to finish |
When this is off, the action succeeds as soon as Airflow accepts the run. |
Check every (seconds) |
How often to ask Airflow for the state of the run. Default: 10. |
Timeout (seconds) |
The action fails when the run has not finished after this many seconds. 0, the default, waits as long as it takes. |
When Airflow does not answer a check, for example while its API server restarts, the action tries again at the next check. It fails after 5 failed checks in a row.
When the wait times out, the checks fail or the workflow is stopped, the DAG run keeps running in Airflow.
Result
| Option | Description |
|---|---|
DAG run id variable |
The variable that receives the id of the new run. Default: |
State variable |
The variable that receives the last known state of the run: |
The action sets both variables in the workflow, so the actions that follow can use them.
Notes
-
The action checks that the DAG exists before it starts a run. A run of a paused DAG stays queued until someone unpauses the DAG. So when the DAG is paused and Wait for the DAG run to finish is on, the action fails without starting a run. When the action does not wait, it starts the run and writes a line about the paused DAG to the log.
-
When a run fails, the log names the tasks that failed.