Hop Tools

Hop contains a number of tools that are optimized for the various tasks Hop developers and administrators need to perform.

Hop GUI

Hop Gui is the visual IDE where Hop data developers create, test, run and manage the life cycle for workflows and pipelines. In addition to functionality for development and life cycle management, Hop Gui contains tools and perspectives to manage projects and environments, to search and manage metadata, to manage and version control a large variety of files and to explore logging in a Neo4j graph.

Hop Gui is designed to be platform independent.
Hop Web is a version of Hop Gui that works on browsers and mobile devices.

The command below starts Hop GUI for your operating system.

Windows
Linux, macOS

hop-gui.bat

./hop-gui.sh

Expected output: Hop Gui starts.

Hop Conf

Hop Conf is a command line tool to manage various aspects of your Hop configuration: projects, environments, cloud configuration and more

The example hop-conf command below lists your available projects.

Windows
Linux, macOS

hop-conf.bat -pl

Expected output:

C:\Program Files\Microsoft\jdk-11.0.17.8-hotspot\\bin\java" -classpath lib\core\*;lib\beam\*;lib\swt\win64\* -Djava.library.path=lib\core;lib\beam -Xmx2048m -DHOP_AUDIT_FOLDER=.\audit -DHOP_PLATFORM_OS=Windows -DHOP_PLATFORM_RUNTIME=Conf -DHOP_AUTO_CREATE_CONFIG=Y org.apache.hop.config.HopConfig  -pl
===[Starting HopConfig]=========================================================
Projects:
  default : config/projects/default
    Configuration file: file:\C:\<YOUR_PATH>\hop\config\projects\default\project-config.json
  samples : config/projects/samples
    Parent project: default
    Configuration file: file:\C:\<YOUR_PATH>\hop\config\projects\samples\project-config.json

./hop-conf.sh -pl

Expected output:

Projects:
  default : config/projects/default
    Configuration file: file:<YOUR_PATH>/hop/config/projects/default/project-config.json
  samples : config/projects/samples
    Parent project: default
    Configuration file: file:<YOUR_PATH>/hop/config/projects/samples/project-config.json

See the Hop Tools → Hop Conf documentation for more information.

Hop Encrypt

Hop Encrypt is a command line tool that obfuscates or encrypts a plain text password for use in XML, password or metadata files. Make sure to also copy the password encryption prefix to indicate the obfuscated nature of the password. Hop will then be able to make the distinction between regular plain text passwords and obfuscated ones.

The example below prints the encrypted Hop password mypassword.

Windows
Linux, macOS

hop-encrypt.bat -hop mypassword

Expected output:

===[Environment Settings - hop-encrypt.bat]====================================
Java identified as "C:\Program Files\Microsoft\jdk-11.0.17.8-hotspot\\bin\java"
HOP_OPTIONS=-Xmx64m -DHOP_AUDIT_FOLDER=.\audit -DHOP_PLATFORM_OS=Windows -DHOP_PLATFORM_RUNTIME=GUI
Command to start Hop will be:
"C:\Program Files\Microsoft\jdk-11.0.17.8-hotspot\\bin\java" -classpath lib\core\*;lib\beam\*;lib\swt\win64\* -Djava.library.path=lib\core;lib\beam -Xmx64m -DHOP_AUDIT_FOLDER=.\audit -DHOP_PLATFORM_OS=Windows -DHOP_PLATFORM_RUNTIME=GUI org.apache.hop.core.encryption.Encr  -hop mypassword
===[Starting HopEncrypt]=========================================================
Encrypted 2be98afc86aa79f9dbb18bd63c99dbdde

./hop-encrypt.sh -hop mypassword

Expected output:

Encrypted 2be98afc86aa79f9dbb18bd63c99dbdde

See the Hop Tools → Hop Encrypt documentation for more information.

Hop Run

Hop Run is a command line tool to run workflows and pipelines, with options to (list or) specify projects, environments, properties and run configurations.

The example below runs the switch-case-basic.hpl pipeline in the samples project (-j) with the local native pipeline run configuration (-r).

Windows
Linux, macOS

hop-run.bat -r local -j samples -f ${PROJECT_HOME}/transforms/switch-case-basic.hpl

Expected output:

C:\<YOUR_PATH>\hop>echo off
===[Environment Settings - hop-run.bat]===================================
Java identified as "C:\Program Files\Microsoft\jdk-11.0.17.8-hotspot\\bin\java"
HOP_OPTIONS="-Xmx2048m" -DHOP_AUDIT_FOLDER=.\audit -DHOP_PLATFORM_OS=Windows -DHOP_PLATFORM_RUNTIME=Run -DHOP_AUTO_CREATE_CONFIG=Y
Consolidated parameters to pass to HopRun are
-r local -j samples -f ${PROJECT_HOME}/transforms/switch-case-basic.hpl
Command to start HopRun will be:
"C:\Program Files\Microsoft\jdk-11.0.17.8-hotspot\\bin\java" -classpath lib\core\*;lib\beam\*;lib\swt\win64\*
-Djava.library.path=lib\core;lib\beam "-Xmx2048m" -DHOP_AUDIT_FOLDER=.\audit
-DHOP_PLATFORM_OS=Windows -DHOP_PLATFORM_RUNTIME=Run -DHOP_AUTO_CREATE_CONFIG=Y org.apache.hop.run.HopRun
-r local -j samples -f ${PROJECT_HOME}/transforms/switch-case-basic.hpl
===[Starting HopRun]=========================================================
2022/12/16 07:07:47 - HopRun - Enabling project 'samples'
2022/12/16 07:07:47 - HopRun - Starting pipeline: config/projects/samples/transforms/switch-case-basic.hpl
2022/12/16 07:07:47 - switch-case-basic - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
2022/12/16 07:07:47 - switch-case-basic - Execution started for pipeline [switch-case-basic]
2022/12/16 07:07:47 - Test Data.0 - Finished processing (I=0, O=0, R=0, W=5, U=0, E=0)
2022/12/16 07:07:47 - Switch id.0 - Finished processing (I=0, O=0, R=5, W=5, U=0, E=0)
2022/12/16 07:07:47 - Output 2.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/16 07:07:47 - Output 1.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/16 07:07:47 - Output 4.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/16 07:07:47 - Output default.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/16 07:07:47 - Output 3.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/16 07:07:47 - switch-case-basic - Pipeline duration : 0.435 seconds [  0.435" ]

./hop-run.sh -r local -j samples -f ${PROJECT_HOME}/transforms/switch-case-basic.hpl

Expected output:

2022/12/16 06:59:03 - HopRun - Enabling project 'samples'
2022/12/16 06:59:03 - HopRun - Relative path filename specified: config/projects/samples//transforms/switch-case-basic.hpl
2022/12/16 06:59:03 - HopRun - Starting pipeline: config/projects/samples//transforms/switch-case-basic.hpl
2022/12/16 06:59:03 - switch-case-basic - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
2022/12/16 06:59:03 - switch-case-basic - Execution started for pipeline [switch-case-basic]
2022/12/16 06:59:04 - Test Data.0 - Finished processing (I=0, O=0, R=0, W=5, U=0, E=0)
2022/12/16 06:59:04 - Switch id.0 - Finished processing (I=0, O=0, R=5, W=5, U=0, E=0)
2022/12/16 06:59:04 - Output default.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/16 06:59:04 - Output 1.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/16 06:59:04 - Output 4.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/16 06:59:04 - Output 3.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/16 06:59:04 - Output 2.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
2022/12/16 06:59:04 - switch-case-basic - Pipeline duration : 0.668 seconds [  0.668" ]

Hop Tools → Hop Run

Hop Search

Hop Search is a command line tool to search all metadata available in a specific project or environment.

The example below searches in the samples project -j for switch-case.

Windows
Linux, macOS

hop-search.bat -j samples -x switch-case

Expected output:

C:\<YOUR_PATH>\hop>echo off
===[Environment Settings - hop-search.bat]===================================
Java identified as "C:\Program Files\Microsoft\jdk-11.0.17.8-hotspot\\bin\java"
HOP_OPTIONS=-Xmx2048m -DHOP_AUDIT_FOLDER=.\audit -DHOP_PLATFORM_OS=Windows -DHOP_PLATFORM_RUNTIME=Search
-DHOP_AUTO_CREATE_CONFIG=Y
Command to start Hop will be:
"C:\Program Files\Microsoft\jdk-11.0.17.8-hotspot\\bin\java" -classpath lib\core\*;lib\beam\*;lib\swt\win64\*
-Djava.library.path=lib\core;lib\beam -Xmx2048m -DHOP_AUDIT_FOLDER=.\audit -DHOP_PLATFORM_OS=Windows
-DHOP_PLATFORM_RUNTIME=Search -DHOP_AUTO_CREATE_CONFIG=Y org.apache.hop.config.HopConfig  -j samples -x switch-case
===[Starting HopConfig]=========================================================
Enabling project 'samples'
Searching in project : samples
Searching for [switch-case]  Case sensitive? true  Regular expression? true
Searching in location : Project samples
-----------------------------------------------------------------------------------
file:///C:/<YOUR_PATH>/hop/config/projects/samples/beam/pipelines/switch-case.hpl : null(switch-case) : matching property value: switch-case    file:///C:/<YOUR_PATH>/hop/config/projects/samples/beam/pipelines/switch-case.hpl : switch-case(switch-case) : matching property value: switch-case
file:///C:/<YOUR_PATH>/hop/config/projects/samples/beam/pipelines/switch-case.hpl : switch-case(switch-case) : pipeline transform property : filePrefix

./hop-search.sh -j samples -x switch-case

Expected output:

Enabling project 'samples'
Searching in project : samples
Searching for [switch-case]  Case sensitive? true  Regular expression? true
Searching in location : Project samples
-----------------------------------------------------------------------------------
file://<YOUR_PATH>/hop/config/projects/samples/beam/pipelines/switch-case.hpl : null(switch-case) : matching property value: switch-case
file://<YOUR_PATH>/hop/config/projects/samples/beam/pipelines/switch-case.hpl : switch-case(switch-case) : matching property value: switch-case
file://<YOUR_PATH>/hop/config/projects/samples/beam/pipelines/switch-case.hpl : switch-case(switch-case) : pipeline transform property : filePrefix

See the Hop Tools → Hop Search documentation for more information.

Hop Server

Hop Server is a web service interface to manage and run workflows and pipelines.

The example command below starts the most basic server on your machine localhost on port 8081.

Windows
Linux, macOS

hop-server.bat localhost 8081

Expected output:

C:\<YOUR_PATH\hop>echo off
===[Environment Settings - hop-server.bat]====================================
Java identified as "C:\Program Files\Microsoft\jdk-11.0.17.8-hotspot\\bin\java"
HOP_OPTIONS=-Xmx2048m -DHOP_AUDIT_FOLDER=.\audit -DHOP_PLATFORM_OS=Windows -DHOP_PLATFORM_RUNTIME=GUI -DHOP_AUTO_CREATE_CONFIG=Y
Command to start Hop will be:
"C:\Program Files\Microsoft\jdk-11.0.17.8-hotspot\\bin\java" -classpath lib\core\*;lib\beam\*;lib\swt\win64\*
-Djava.library.path=lib\core;lib\beam -Xmx2048m -DHOP_AUDIT_FOLDER=.\audit -DHOP_PLATFORM_OS=Windows
-DHOP_PLATFORM_RUNTIME=GUI -DHOP_AUTO_CREATE_CONFIG=Y org.apache.hop.www.HopServer  localhost 8081
===[Starting HopServer]=========================================================
2022/12/16 07:33:44 - HopServer - Enabling project 'default'
2022/12/16 07:33:44 - HopServer - Installing timer to purge stale objects after 1440 minutes.
2022/12/16 07:33:44 - HopServer - Created listener for webserver @ address : localhost:8081

Your Hop Server is available at http://localhost:8081, log on with the default username cluster, password cluster.

Hit CTRL-C to stop the server.

./hop-server.sh localhost 8081

Expected output:

2022/12/16 07:20:19 - HopServer - Enabling project 'default'
2022/12/16 07:20:19 - HopServer - Installing timer to purge stale objects after 1440 minutes.
2022/12/16 07:20:19 - HopServer - Created listener for webserver @ address : localhost:8081

Your Hop Server is available at http://localhost:8081, log on with the default username cluster, password cluster.

Hit CTRL-C to stop the server.

Hop Import

Hop Import is a command line tool to import PDI/Kettle jobs and transformations into Apache Hop.

Hop Import does a lot more than just a code conversion from jobs to workflows and transformations to pipelines: database connections are converted to Apache Hop rdbms connections, variables are parsed and imported, and everything is bundled into a full Apache Hop project.

The example command below prints the hop-import help. Check the Import Kettle (PDI) Projects in Apache Hop docs for more information.

Windows
Linux, macOS

hop-import.bat

./hop-import.sh