Hop Run

Hop Run is a command line tool to run your workflows and pipelines.

Hop Run can be launched through the script with the corresponding name in your Hop installation directory, either hop-run.sh on Mac and Linux or hop-run.bat on Windows.

Except for the launcher script, all options are platform agnostic; there is no difference in the way hop-run works on any operating system.

Hop Run Options

You can display Hop Run’s options by running the command without any options.

On Windows, run hop-run.bat. The command on Mac and Linux is hop-run.sh.

You’ll see the option listed in a similar output to the one below:

Usage

Output of help

bash

Usage: <main class> [-ho] [-e=<environmentOption>] [-f=<filename>]
                    [-j=<projectOption>] [-l=<level>]
                    [-r=<runConfigurationName>] [-ps=<parametersSeparator] [-p=<parameters>[,
                    <parameters>...]]... [-s=<systemProperties>[,
                    <systemProperties>...]]...
  -a, --startaction=<startActionName>
                          The name of the action where to start a workflow
  -au, --allow-unsupported
                          Allow running pipelines or workflows that contain
                          transforms or actions marked UNSUPPORTED on the
                          selected engine. Strict by default; this flag is the
                          CLI equivalent of the GUI "Run anyway" override.
  -e, --environment=<environmentOption>
                          The name of the lifecycle environment to use
  -f, --file=<filename>   The filename of the workflow or pipeline to run
  -h, --help              Displays this help message and quits
  -j, --project=<projectOption>
                          The name of the project to use
  -l, --level=<level>     The debug level, one of NOTHING, ERROR, MINIMAL, BASIC, DETAILED, DEBUG, ROWLEVEL
  -l, --level=<level>     The debug level, one of NOTHING, ERROR, MINIMAL, BASIC, DETAILED, DEBUG, ROWLEVEL
  -lf, --logfile=<logfile-name> The complete filename where hop-run will write the Hop console log.
                          An existing file is replaced unless --logfile-append is set
  -lfa, --logfile-append  Append to the file given by --logfile instead of replacing it.
                          Without this option an existing log file is overwritten.
                          Ignored when --logfile is not set
  -m, --metadata-export=<metadataExportFile>
                          A file containing exported metadata in JSON format
  -o, --printoptions      Print the used options
  -p, --parameters=<parameters>[,<parameters>...]
                          A list of PARAMETER=VALUE pairs
  -ps, --parameters-separator=<parametersSeparator>
                          A character to be used as separator for our list of PARAMETER=VALUE pairs (default is ",").
  -r, --runconfig=<runConfigurationName>
                          The name of the Run Configuration to use
  -s, --system-properties=<systemProperties>[,<systemProperties>...]
                          A comma separated list of KEY=VALUE pairs
  -swt, --shutdown-timeout=<shutdownTimeout>
                          The maximum number of seconds to wait for the pipeline or workflow to stop
                          when hop-run receives a termination signal (default 5)
  -v, --version           Print version information and exit

The available options are listed in more detail in the table below:

Short Extended Description

Short	Extended	Description
`-e`	`--environment`	The name of the environment to use. The project name is taken from that environment, so you do not need `-j` as well. Check the documentation on environments for more details.
`-a`	`--startaction`	The name of the action where to start a workflow
`-au`	`--allow-unsupported`	Bypass the engine-compatibility gate and run pipelines or workflows that contain transforms or actions marked UNSUPPORTED on the selected engine. Strict by default — when omitted, Hop Run refuses to start such a run and prints the list of incompatible elements. This flag is the CLI equivalent of the "Run anyway" button in the GUI run dialog. Under the hood it sets the `HOP_ALLOW_UNSUPPORTED=Y` variable on the execution, which propagates to every nested child pipeline / workflow so the override applies to the full call tree.
`-f`	`--file`	The filename of the workflow or pipeline to run
`-h`	`--help`	Displays this help message and quits.
`-j`	`--project`	The name of the project to use when running a pipeline or workflow. Ignored when `-e` is set: the project of that environment is used. To use a workflow of another project, see Run a workflow from another project.
`-l`	`--level`	The debug level, one of NOTHING, ERROR, MINIMAL, BASIC, DETAILED, DEBUG, ROWLEVEL
`-lf`	`--logfile`	The complete filename where hop-run will write the Hop console log. An existing file is replaced unless `--logfile-append` is also set.
`-lfa`	`--logfile-append`	Append to the file given by `--logfile` instead of replacing it. Without this option an existing log file is overwritten. Ignored when `--logfile` is not set. Use this when several executions should share one file, for example one log per day.
`-m`	`--metadata-export`	A file containing exported metadata in JSON format. See also the metadata export option in Hop Conf
`-o`	`--printoptions`	Print the used options
`-p`	`--parameters`	A comma separated list of PARAMETER=VALUE pairs. See Working with parameters for which value a parameter gets.
`-ps`	`--parameters-separator`	A character to be used as separator for our list of PARAMETER=VALUE pairs (default is ",").
`-r`	`--runconfig`	The name of the Run Configuration to use. Check the documentation on run configurations for more details.
`-s`	`--system-properties`	A comma separated list of KEY=VALUE pairs
`-swt`	`--shutdown-timeout`	The maximum number of seconds to wait for the pipeline or workflow to stop when Hop Run receives a termination signal. The default is 5 seconds. Can also be set with the `HOP_RUN_SHUTDOWN_TIMEOUT` environment variable. See Stopping Hop Run.
`-ime`	`--in-memory`	Keep Hop configuration in memory. Changes are not written to `hop-config.json`. Implied by `--project-locations` and `--environments`.
`-pl`	`--project-locations`	Comma-separated project locations to register for this process only (`name=/path` or a zip/jar). See Use a project without hop-config.json.
	`--environments`	Comma-separated environment definitions (`name=[project:]file1;file2`) registered in memory for this process.
	`--environment-conf-files` / `--environment-config-files`	Extra environment configuration files to apply to the enabled project or environment.

-e

--environment

The name of the environment to use. The project name is taken from that environment, so you do not need -j as well.

Check the documentation on environments for more details.

-a

--startaction

The name of the action where to start a workflow

-au

--allow-unsupported

Bypass the engine-compatibility gate and run pipelines or workflows that contain transforms or actions marked UNSUPPORTED on the selected engine.

Strict by default — when omitted, Hop Run refuses to start such a run and prints the list of incompatible elements. This flag is the CLI equivalent of the "Run anyway" button in the GUI run dialog. Under the hood it sets the HOP_ALLOW_UNSUPPORTED=Y variable on the execution, which propagates to every nested child pipeline / workflow so the override applies to the full call tree.

-f

--file

The filename of the workflow or pipeline to run

-h

--help

Displays this help message and quits.

-j

--project

The name of the project to use when running a pipeline or workflow. Ignored when -e is set: the project of that environment is used. To use a workflow of another project, see Run a workflow from another project.

-l

--level

The debug level, one of NOTHING, ERROR, MINIMAL, BASIC, DETAILED, DEBUG, ROWLEVEL

-lf

--logfile

The complete filename where hop-run will write the Hop console log. An existing file is replaced unless --logfile-append is also set.

-lfa

--logfile-append

Append to the file given by --logfile instead of replacing it. Without this option an existing log file is overwritten. Ignored when --logfile is not set. Use this when several executions should share one file, for example one log per day.

-m

--metadata-export

A file containing exported metadata in JSON format. See also the metadata export option in Hop Conf

-o

--printoptions

Print the used options

-p

--parameters

A comma separated list of PARAMETER=VALUE pairs. See Working with parameters for which value a parameter gets.

-ps

--parameters-separator

A character to be used as separator for our list of PARAMETER=VALUE pairs (default is ",").

-r

--runconfig

The name of the Run Configuration to use. Check the documentation on run configurations for more details.

-s

--system-properties

A comma separated list of KEY=VALUE pairs

-swt

--shutdown-timeout

The maximum number of seconds to wait for the pipeline or workflow to stop when Hop Run receives a termination signal. The default is 5 seconds. Can also be set with the HOP_RUN_SHUTDOWN_TIMEOUT environment variable. See Stopping Hop Run.

-ime

--in-memory

Keep Hop configuration in memory. Changes are not written to hop-config.json. Implied by --project-locations and --environments.

-pl

--project-locations

Comma-separated project locations to register for this process only (name=/path or a zip/jar). See Use a project without hop-config.json.

--environments

Comma-separated environment definitions (name=[project:]file1;file2) registered in memory for this process.

--environment-conf-files / --environment-config-files

Extra environment configuration files to apply to the enabled project or environment.

Possible exit codes

Hop Run currently finishes with one of four exit codes.

In all cases except 0, check the error logs for more detailed information for your specific error.

Exit code	Error Message	Description
0	None	No error message. Everything worked flawlessly.
1	Error found during execution!	There was an error executing the workflow or pipeline.
2	General error found, something went horribly wrong!	There was a general error that caused Hop Run to fail completely.
9	None	There was an error parsing the provided parameter information. Hop Run shows the syntax summary and exists.
143	None	Hop Run received SIGTERM and stopped the run, see Stopping Hop Run. A process that receives SIGINT (Ctrl-C) or SIGHUP ends with 130 or 129.

Exit code

Error Message

Description

None

No error message. Everything worked flawlessly.

Error found during execution!

There was an error executing the workflow or pipeline.

General error found, something went horribly wrong!

There was a general error that caused Hop Run to fail completely.

None

There was an error parsing the provided parameter information. Hop Run shows the syntax summary and exists.

143

None

Hop Run received SIGTERM and stopped the run, see Stopping Hop Run. A process that receives SIGINT (Ctrl-C) or SIGHUP ends with 130 or 129.

Stopping Hop Run

When Hop Run receives a termination signal (SIGTERM, SIGINT or SIGHUP) it stops the running pipeline or workflow the same way the Stop button in Hop GUI does, and waits for the run to end before the process exits. This happens when you press Ctrl-C, run docker stop, or when Kubernetes or OpenShift evicts a pod, for example while it drains a node. Transforms and actions stop and release their resources (database connections, open files, …), and the log shows that the run was stopped.

Hop Run waits at most the number of seconds set with --shutdown-timeout (or the HOP_RUN_SHUTDOWN_TIMEOUT environment variable), 5 seconds by default, and then exits anyway. When lineage is enabled, Hop Run then delivers the lineage events of the run, including its end, before it exits.

Keep the timeout below the time the platform gives a process before it kills it: 30 seconds by default in Kubernetes (terminationGracePeriodSeconds) and 10 seconds for docker stop (docker stop --time). Leave room for the lineage delivery as well. To give a run more time to stop, raise both: for example HOP_RUN_SHUTDOWN_TIMEOUT=60 with docker run --stop-timeout 90 (stop_grace_period in Docker Compose) or terminationGracePeriodSeconds: 90.

SIGKILL (kill -9, or the platform giving up) cannot be handled: the process ends at once, in the middle of whatever it was doing.

Stopping a run does not undo the work it did so far. Design pipelines and workflows that may be stopped so that they can be restarted.

Examples

The example below runs the switch-case-basic.hpl pipeline from the samples project through hop-run.

Windows
Linux, macOS

Open a command (CMD) window, change to the directory where you unzipped Apache Hop and run:

hop-run.bat -j samples -r local -f ${PROJECT_HOME}/transforms/switch-case-basic.hpl

Your output will be similar to what is shown below:

"C:\Program Files\Microsoft\jdk-11.0.17.8-hotspot\\bin\java" -classpath lib\core\*;lib\beam\*;lib\swt\win64\*
-Djava.library.path=lib\core;lib\beam "-Xmx2048m" -DHOP_AUDIT_FOLDER=.\audit
-DHOP_PLATFORM_OS=Windows -DHOP_PLATFORM_RUNTIME=Run -DHOP_AUTO_CREATE_CONFIG=Y org.apache.hop.run.HopRun
-j samples -r local -f ${PROJECT_HOME}/transforms/switch-case-basic.hpl
===[Starting HopRun]=========================================================
2022/12/12 14:09:58 - HopRun - Enabling project 'samples'
2022/12/12 14:09:58 - HopRun - Starting pipeline: config/projects/samples/transforms/switch-case-basic.hpl
2022/12/12 14:09:58 - switch-case-basic - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
2022/12/12 14:09:58 - switch-case-basic - Execution started for pipeline [switch-case-basic]
2022/12/12 14:09:58 - Test Data.0 - Finished processing (I=0, O=0, R=0, W=5, U=0, E=0)
2022/12/12 14:09:58 - Switch id.0 - Finished processing (I=0, O=0, R=5, W=5, U=0, E=0)
2022/12/12 14:09:58 - Output 2.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/12 14:09:58 - Output default.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/12 14:09:58 - Output 1.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/12 14:09:58 - Output 4.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/12 14:09:58 - Output 3.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/12 14:09:58 - switch-case-basic - Pipeline duration : 0.542 seconds [  0.542" ]

Open a terminal, change to the directory where you unzipped Apache Hop and run:

./hop-run.sh -j samples -r local -f ${PROJECT_HOME}/transforms/switch-case-basic.hpl

Your output will be similar to what is shown below:

2022/12/12 14:06:39 - HopRun - Enabling project 'samples'
2022/12/12 14:06:39 - HopRun - Relative path filename specified: config/projects/samples//transforms/switch-case-basic.hpl
2022/12/12 14:06:39 - HopRun - Starting pipeline: config/projects/samples//transforms/switch-case-basic.hpl
2022/12/12 14:06:39 - switch-case-basic - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
2022/12/12 14:06:39 - switch-case-basic - Execution started for pipeline [switch-case-basic]
2022/12/12 14:06:39 - Test Data.0 - Finished processing (I=0, O=0, R=0, W=5, U=0, E=0)
2022/12/12 14:06:39 - Switch id.0 - Finished processing (I=0, O=0, R=5, W=5, U=0, E=0)
2022/12/12 14:06:39 - Output 2.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/12 14:06:39 - Output default.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/12 14:06:39 - Output 4.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/12 14:06:39 - Output 3.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/12 14:06:39 - Output 1.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/12 14:06:39 - switch-case-basic - Pipeline duration : 0.499 seconds [  0.499" ]

To run a project folder that is not registered in hop-config.json:

bash

hop --project-locations my-project=/path/to/project run -f pipeline.hpl -r local

See Use a project without hop-config.json for the location and environment formats.

Running a pipeline with a parameter

The pipelines/pipeline-with-parameter.hpl pipeline in the samples project has a parameter PRM_EXAMPLE without a default value. It writes the value it received to the log, followed by the value of the variable ENV_VARIABLE.

Give the parameter a value with -p (--parameters). The single quotes keep the shell from replacing ${PROJECT_HOME}, so Hop gets it as it is. On Windows, use hop-run.bat instead of ./hop-run.sh and leave out the single quotes: CMD doesn’t remove them.

./hop-run.sh -j samples -r local -f '${PROJECT_HOME}/pipelines/pipeline-with-parameter.hpl' -p PRM_EXAMPLE=Hello

The log includes these lines, with the value of the parameter:

2026/11/02 12:57:45 - HopRun - Enabling project 'samples'
2026/11/02 12:57:45 - HopRun - Starting pipeline: config/projects/samples/pipelines/pipeline-with-parameter.hpl
2026/11/02 12:57:45 - pipeline-with-parameter - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
2026/11/02 12:57:45 - pipeline-with-parameter - Execution started for pipeline [pipeline-with-parameter]
2026/11/02 12:57:45 - get ${PRM_EXAMPLE}.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2026/11/02 12:57:45 - write parameter to log.0 -
2026/11/02 12:57:45 - write parameter to log.0 - ------------> Linenr 1------------------------------
2026/11/02 12:57:45 - write parameter to log.0 - we received 'Hello' as the value for this pipeline's parameter.
2026/11/02 12:57:45 - write parameter to log.0 -
2026/11/02 12:57:45 - write parameter to log.0 - example = Hello
2026/11/02 12:57:45 - write parameter to log.0 -
2026/11/02 12:57:45 - write parameter to log.0 - ====================
2026/11/02 12:57:45 - write env_variable to log.0 -
2026/11/02 12:57:45 - write env_variable to log.0 - ------------> Linenr 1------------------------------
2026/11/02 12:57:45 - write env_variable to log.0 - the value for variable ENV_VARIABLE is ${ENV_VARIABLE}
2026/11/02 12:57:45 - write env_variable to log.0 -
2026/11/02 12:57:45 - write env_variable to log.0 - env_variable = ${ENV_VARIABLE}
2026/11/02 12:57:45 - write env_variable to log.0 -
2026/11/02 12:57:45 - write env_variable to log.0 - ====================
2026/11/02 12:57:45 - write parameter to log.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2026/11/02 12:57:45 - get ${ENV_VARIABLE}.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2026/11/02 12:57:45 - write env_variable to log.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2026/11/02 12:57:45 - pipeline-with-parameter - Pipeline duration : 0.039 seconds [  0.039" ]

Without -p, the parameter gets its default value, which is empty here: the log shows we received '' as the value for this pipeline’s parameter.

ENV_VARIABLE isn’t set in the samples project, so the log shows ${ENV_VARIABLE} as it is. Set it, for example, as a system property with -s, or in an environment. A value with spaces goes between quotes (double quotes on Windows):

./hop-run.sh -j samples -r local -f '${PROJECT_HOME}/pipelines/pipeline-with-parameter.hpl' -p 'PRM_EXAMPLE=Hello Hop' -s ENV_VARIABLE=dev

... - write parameter to log.0 - we received 'Hello Hop' as the value for this pipeline's parameter.
... - write env_variable to log.0 - the value for variable ENV_VARIABLE is dev

See Working with parameters for more on parameters.

Parameter Examples

This is a list of examples on how the parameters on this command are parsed

Normal Usage

--parameters=key1=value1,key2=value2

Result:

Key	value
key1	value1
key2	value2

Key

value

key1

value1

key2

value2

Spaces in value Usage

--parameters=key1="This value contains spaces",key2=value2

Result:

Key	value
key1	This value contains spaces
key2	value2

Key

value

key1

This value contains spaces

key2

value2

Commas in value Usage

--parameters=key1=\"value1,value2\"

Result:

Key	value
key1	value1,value2

Forcing quotes in value

--parameters=key1="\"\"String with spaces\"\""

Result:

Key	value
key1	"String with spaces"

Scheduling

Hop Run is the command a scheduler should invoke: it runs one file and exits. See Scheduling workflows and pipelines for cron, at, systemd, Windows Task Scheduler, Jenkins, Kubernetes CronJob and Apache Airflow.