Databricks VFS Connection

Description

Databricks VFS Connection registers a named VFS scheme for Unity Catalog Volumes and Workspace files via the Databricks Files API.

Example: VFS name db-volume, root path /Volumes/apache-hop/default/testing, Databricks Connection prod-workspace:

db-volume:///input/customers-1M.txt

resolves to workspace path /Volumes/apache-hop/default/testing/input/customers-1M.txt.

This is Hop-side I/O only (GUI, local engine, classic file transforms). Native Spark Dataset I/O does not use Hop VFS schemes — use absolute /Volumes/… or cloud URIs (s3a://…, …) on the cluster. Do not use this connection as the primary bulk path for large Spark datasets (workspace Files API is not a high-throughput object-store client). See Databricks Volumes / Workspace VFS and Paths and file systems on native Spark.

Options

Option Description

Name (URI scheme)

VFS scheme used in paths: {name}:///…. Prefer short names; do not reuse built-in schemes (file, s3, …).

Description

Optional free-text description.

Databricks Connection

Existing Jobs/workspace connection that supplies host and personal access token.

Root path

Absolute UC Volume or Workspace path that is the root of this scheme. Shortens URIs: {name}:///input → {root}/input. Leave empty if every URI must include full /Volumes/… or /Workspace/… paths.

Tips

  • Create the Databricks Connection first and use Test connection there (or on this editor, which validates the referenced connection).

  • Set Root path to your testing volume (for example /Volumes/apache-hop/default/testing) so workflow paths stay short.

  • In File → Open, type {name}:/// to list the root path contents (when root is set).

  • One Jobs connection can back several VFS schemes (different root volumes).

Related