Hop

The hop command line tool is simplification of the many scripts that the project ships with like 'hop-run`, hop-search, hop-gui, and so on. The main goal is to have a single wrapper script to make maintenance easier for the developers, but also to allow plugins to contribute new commands.

Usage

== Usage

To see the usage of the hop tool, you can simply use the help command to see which commands are available. The command list depends on the plugins in your installation; the sample below is the standard Hop client, without optional plugins.

bash

$ ./hop help
Usage: hop [-hV] [--dev-debug] [--dev-debug-wait] [-ime]
           [-e=<environmentOption>] [-j=<projectOption>]
           [--environment-conf-files=<environmentConfigFiles>[,
           <environmentConfigFiles>...]]... [--environments=<environments>[,
           <environments>...]]... [-pl=<projectLocations>[,
           <projectLocations>...]]... [-s=<systemProperties>[,
           <systemProperties>...]]... [COMMAND]
      --dev-debug         Allow a Hop developer to debug remotely
      --dev-debug-wait    Allow a Hop developer to debug remotely. The script
                            will wait until a debugging session is set up.
  -e, --environment=<environmentOption>
                          The name of the lifecycle environment to use
      --environment-conf-files,
        --environment-config-files=<environmentConfigFiles>[,
        <environmentConfigFiles>...]
                          Comma-separated list of configuration files to apply
                            to the project or environment
      --environments=<environments>[,<environments>...]
                          Comma-separated list of environment definitions (name=
                            [project:]configFile1;configFile2)
  -h, --help              Show this help message and exit.
      -ime, --in-memory   Keep configuration in memory without persisting to
                            hop-config.json
  -j, --project=<projectOption>
                          The name of the project to use
      -pl, --project-locations=<projectLocations>[,<projectLocations>...]
                          Comma-separated list of project locations
                            (name=location or archive files)
  -s, --system-properties=<systemProperties>[,<systemProperties>...]
                          A comma separated list of KEY=VALUE pairs
  -V, --version           Print version information and exit.
Commands:
  help          Display help information about the specified command.
  conf          Configure Hop
  driver        Manage JDBC drivers (list, install)
  encrypt       Encrypt secrets
  export        Export pipeline, workflow, or model diagrams to various formats
                  (SVG, Mermaid, etc.)
  gui           The Hop GUI
  import        Import metadata
  marketplace   Install and manage optional Hop plugins from a Maven repository
  naming-check  Validate names against project Naming Schemes
  run           Run a pipeline or workflow
  search        Search in Hop metadata
  server        Run a Hop server
  setup         Configure Hop launcher environment variables
                  (HOP_CONFIG_FOLDER, HOP_AUDIT_FOLDER, HOP_JAVA_HOME,
                  HOP_OPTIONS, HOP_SHARED_JDBC_FOLDERS). Distinct from hop
                  conf, which edits hop-config.json.

Commands

Command Plugin Page

Command	Plugin	Page
`arrow`	Apache Arrow (optional)	hop arrow
`conf`		hop conf
`doc`	Documentation (optional)	hop doc
`driver`		hop driver
`encrypt`		hop encrypt
`export`		hop export
`gui`		hop gui
`import`		hop import
`lint`	Lint (optional)	hop lint
`marketplace`		hop marketplace
`naming-check`		Naming scheme: checking names in CI
`python`	PyHop (optional)	hop python
`run`		hop run
`search`		hop search
`server`		hop server
`setup`		hop setup

arrow

Apache Arrow (optional)

hop arrow

conf

hop conf

doc

Documentation (optional)

driver

encrypt

export

gui

import

lint

Lint (optional)

hop lint

marketplace

hop marketplace

naming-check

Naming scheme: checking names in CI

python

PyHop (optional)

run

search

server

setup

The commands marked optional only exist once their plugin is installed from the marketplace. hop help then lists them as well.

Options

Short Extended Description

Short	Extended	Description
	`--dev-debug`	For Hop developers: let a Java debugger connect on port 5010.
	`--dev-debug-wait`	As `--dev-debug`, but Hop waits until the debugger is connected before it starts.
`-e`	`--environment`	The name of the lifecycle environment to enable before the subcommand runs.
`-h`	`--help`	Displays this help message and quits.
`-ime`	`--in-memory`	Keep Hop configuration in memory. Changes are not written to `hop-config.json`. Implied by `--project-locations` and `--environments`.
`-j`	`--project`	The name of the project to enable before the subcommand runs.
`-pl`	`--project-locations`	Comma-separated project locations to register for this process only (`name=/path`, a zip/jar, or a bare folder). See Use a project without hop-config.json.
	`--environments`	Comma-separated environment definitions (`name=[project:]file1;file2`) registered in memory for this process.
	`--environment-conf-files` / `--environment-config-files`	Extra environment configuration files to apply to the enabled project or environment.
`-s`	`--system-properties`	A comma separated list of KEY=VALUE pairs, set before Hop starts.
`-V`	`--version`	Show the current Hop version.

--dev-debug

For Hop developers: let a Java debugger connect on port 5010.

--dev-debug-wait

As --dev-debug, but Hop waits until the debugger is connected before it starts.

-e

--environment

The name of the lifecycle environment to enable before the subcommand runs.

-h

--help

Displays this help message and quits.

-ime

--in-memory

Keep Hop configuration in memory. Changes are not written to hop-config.json. Implied by --project-locations and --environments.

-j

--project

The name of the project to enable before the subcommand runs.

-pl

--project-locations

Comma-separated project locations to register for this process only (name=/path, a zip/jar, or a bare folder). See Use a project without hop-config.json.

--environments

Comma-separated environment definitions (name=[project:]file1;file2) registered in memory for this process.

--environment-conf-files / --environment-config-files

Extra environment configuration files to apply to the enabled project or environment.

-s

--system-properties

A comma separated list of KEY=VALUE pairs, set before Hop starts.

-V

--version

Show the current Hop version.

Root options such as --project-locations are processed before the subcommand (run, gui, search, …), so the project is already registered when the command runs:

bash

hop --project-locations my-project=/data/my-project run -f pipeline.hpl -r local
hop --project-locations my-project=/data/my-project gui -j my-project -f pipeline.hpl

Setup

hop setup writes launcher environment variables (HOP_CONFIG_FOLDER, HOP_AUDIT_FOLDER, and related). It is not the same as hop conf, which edits hop-config.json.