Databricks job wait
Description
Databricks job wait polls the Jobs API (jobs/runs/get) until a run reaches a terminal state (SUCCESS, FAILED, …). Use it after Databricks job run in Fire and forget mode, or when another system already started a run and you only have a Run ID.
There is no dedicated “wait” endpoint on Databricks: this action repeatedly GETs run status, sleeps for the poll interval, and exits when the run finishes or the timeout is hit.
Typical pattern
-
Job Run with Fire and forget → sets
${DatabricksRunId} -
Other workflow actions (notifications, parallel prep, …)
-
Job Wait with Run ID
${DatabricksRunId}→ success/failure hops
For “submit and block on this step,” use Job Run’s built-in Wait for completion instead.
Options
| Option | Description |
|---|---|
Databricks connection |
Databricks Connection (PAT) |
Run ID |
Numeric run id; default |
Timeout (seconds, 0=none) |
Maximum wait; |
Poll interval (seconds) |
Delay between status checks (default 15, at least 1) |
Cancel Databricks run if workflow is stopped or the timeout is reached |
If checked, calls |
Result variables |
Same defaults as Job Run: |
Result
What this action passes on to the next action. See Action results for how results travel through a workflow.
| Result | Value |
|---|---|
True |
The run ended with life cycle state |
False |
The number of errors is 1. Databricks connection is empty or doesn’t exist, Run ID is empty or not a number, a call to the Jobs API fails, the run ends with another result state (for example |
Result files |
Passed on unchanged. |
Result rows |
Passed on unchanged. |
After every status check the action sets the result variables for the Job ID, the Run ID, the status (the result state, like SUCCESS or FAILED, once the run is TERMINATED, the life cycle state before that) and the run page URL (the Job ID and the URL only when Databricks returns them), in its workflow and in every parent workflow. The error variable is emptied when the action starts and holds the error message when the action is false.
|