Databricks Job Wait

Description

Databricks Job Wait polls the Jobs API (jobs/runs/get) until a run reaches a terminal state (SUCCESS, FAILED, …). Use it after Databricks Job Run in Fire and forget mode, or when another system already started a run and you only have a Run ID.

There is no dedicated “wait” endpoint on Databricks: this action repeatedly GETs run status, sleeps for the poll interval, and exits when the run finishes or the timeout is hit.

Typical pattern

  1. Job Run with Fire and forget → sets ${DatabricksRunId}

  2. Other workflow actions (notifications, parallel prep, …)

  3. Job Wait with Run ID ${DatabricksRunId} → success/failure hops

For “submit and block on this step,” use Job Run’s built-in Wait for completion instead.

Options

Option Description

Databricks connection

Databricks Connection (PAT)

Run ID

Numeric run id; default ${DatabricksRunId}

Timeout (seconds)

Maximum wait; 0 = no timeout

Poll interval (seconds)

Delay between status checks (default 15)

Cancel run if workflow is stopped

If checked, calls jobs/runs/cancel when the parent workflow stops

Result variables

Same defaults as Job Run: DatabricksJobId, DatabricksRunId, DatabricksStatus, DatabricksRunPageUrl, DatabricksError

Related