Hop Conf - The Hop command line configuration tool

Hop Conf changes the Hop configuration from the command line, so you can script what you otherwise do in Hop Gui: create projects and environments, set variables and change settings.

It writes to hop-config.json, in the folder of the HOP_CONFIG_FOLDER variable or by default in the config folder of your Hop installation, and to the configuration files of projects and environments. Run it as hop-conf.sh, as hop-conf.bat on Windows, or as hop conf with the hop command.

hop-conf.sh -h lists all the options of your installation. Plugins add their own, for example for search, the File Explorer, linting, the AI Assistant or cloud storage. The options described on this page are listed in Options.

Projects and environments

The examples use hop-conf.sh on Linux and macOS. On Windows, run hop-conf.bat with the same options, and put a command on a single line or end its lines with ^ instead of \.

Projects

A project is registered in hop-config.json with its name and its home folder. Its settings and variables are stored in project-config.json, in the home folder.

Create a project

The home folder has to exist: when it doesn’t, hop-conf stops with an error that the home folder does not exist.

bash

mkdir -p /data/projects/sales
./hop-conf.sh --project-create --project=sales --project-home=/data/projects/sales

Output:

shell

Creating project 'sales'
Project 'sales' was created for home folder : /data/projects/sales
Configuration file for project 'sales' was saved to : file:/data/projects/sales/project-config.json

You can set more at the same time, for example a parent project to inherit metadata and variables from (--project-parent), a description (--project-description) or variables (--project-variables, see Project variables).

When the home folder already has a project-config.json, --project-create updates it with the options you pass and keeps its variables. Add --project-keep-config-file to leave the file as it is.

Modify, list and delete projects

--project-modify changes the options you pass, for example ./hop-conf.sh --project-modify --project=sales --project-description="Sales data warehouse".
--projects-list lists the projects with their home folder and configuration file.
--project-delete removes a project from hop-config.json, for example ./hop-conf.sh --project-delete --project=sales. The home folder and its files are not deleted.

Project variables

--project-variables adds variables to the project, or changes the value of a variable that already exists. Variables with other names are kept.

bash

./hop-conf.sh --project-modify --project=sales --project-variables=INPUT_FOLDER=/data/in,OUTPUT_FOLDER=/data/out

Separate variables with commas. To use spaces or commas in a value, wrap the value in double quotes and keep them from the shell:

Option, as typed in a shell Variables

Option, as typed in a shell	Variables
`--project-variables=key1=value1,key2=value2`	`key1` = `value1`, `key2` = `value2`
`--project-variables=key1="This value contains spaces",key2=value2`	`key1` = `This value contains spaces`, `key2` = `value2`
`--project-variables='key1="value1,value2"'`	`key1` = `value1,value2`
`--project-variables=key1=\"value1,value2\"`	`key1` = `value1,value2`
`--project-variables=key1="\"\"String with spaces\"\""`	`key1` = `"String with spaces"`

--project-variables=key1=value1,key2=value2

key1 = value1, key2 = value2

--project-variables=key1="This value contains spaces",key2=value2

key1 = This value contains spaces, key2 = value2

--project-variables='key1="value1,value2"'

key1 = value1,value2

--project-variables=key1=\"value1,value2\"

key1 = value1,value2

--project-variables=key1="\"\"String with spaces\"\""

key1 = "String with spaces"

Environments

An environment belongs to one project. It adds configuration files with variables to that project, usually one environment for every stage of its lifecycle: development, testing, production. The project holds the code, and the environment the configuration, such as server names and folders. See Projects and Environments for how they work together.

Create an environment

bash

./hop-conf.sh --environment-create \
     --environment=sales-dev \
     --environment-project=sales \
     --environment-purpose=Development \
     --environment-config-files=/data/environments/sales-dev/variables.json

Output, shortened:

shell

Creating environment 'sales-dev'
Environment 'sales-dev' was created in Hop configuration file /opt/hop/config/hop-config.json
Created empty environment configuration file : /data/environments/sales-dev/variables.json
  sales-dev
    Purpose: Development
    Project name: sales
      Config file: /data/environments/sales-dev/variables.json

A configuration file that doesn’t exist yet is created, empty. Before creating it, hop-conf logs an error that the file doesn’t exist to read variables from.

json

{ }

Add variables to it as described in Configuration files.

Add configuration files from a directory

--environment-config-files lists files one by one. To add every matching file in a folder, pass the folder and a regular expression. The expression is matched against the file name, the same way as in the JSON Input and Get File Names transforms. .json is not a valid expression; use .\.json.

An exclusion expression and --environment-config-include-subfolders are optional. Files found in the folder are stored as individual paths. They can be combined with --environment-config-files. When either option is set, that selection replaces the environment’s configuration file list.

bash

./hop-conf.sh --environment-create \
     --environment=sales-test \
     --environment-project=sales \
     --environment-purpose=Testing \
     --environment-config-file-directory="/data/environments/sales-test" \
     --environment-config-file-wildcard=".*\.json" \
     --environment-config-file-exclude-wildcard=".*secret.*" \
     --environment-config-include-subfolders

Modify, list and delete environments

--environment-modify changes the options you pass, for example ./hop-conf.sh --environment-modify --environment=sales-dev --environment-purpose=Testing.
--environments-list lists the environments with their purpose, project and configuration files.
--environment-delete removes an environment from hop-config.json, for example ./hop-conf.sh --environment-delete --environment=sales-dev. Its configuration files are not deleted.

Configuration files

The configuration files of an environment hold its variables. --config-file selects the file, --config-file-set-variables sets the value of variables, and --config-file-describe-variables their description:

bash

./hop-conf.sh --config-file=/data/environments/sales-dev/variables.json \
     --config-file-set-variables=DB_HOSTNAME=localhost,DB_PASSWORD=abcd
./hop-conf.sh --config-file=/data/environments/sales-dev/variables.json \
     --config-file-describe-variables="DB_HOSTNAME=The database server"

The file then contains:

json

{
  "variables" : [
    {
      "name" : "DB_HOSTNAME",
      "value" : "localhost",
      "description" : "The database server"
    },
    {
      "name" : "DB_PASSWORD",
      "value" : "abcd",
      "description" : ""
    }
  ]
}

The variables of hop-config.json itself, listed as System variables in the configuration perspective of Hop Gui, are set with --set-variable and described with --describe-variable:

bash

./hop-conf.sh --set-variable=COMPANY_NAME=Acme
./hop-conf.sh --describe-variable="COMPANY_NAME=The company name used in reports"

--set-variable=MY_VARIABLE= sets a variable to an empty value. The variable stays in the list.

Projects plugin configuration

There are various options to configure the behavior of the Projects plugin itself. In Hop configuration file hop-config.json we can find the following options:

json

{
    "projectMandatory" : true,
    "environmentMandatory" : false,
    "defaultProject" : "default",
    "defaultEnvironment" : null,
    "standardParentProject" : "default",
    "standardProjectsFolder" : "/home/matt/test-stuff/"
}

Option Description hop-conf option

Option	Description	hop-conf option
projectMandatory	This will prevent anyone from using hop-run without specifying a project	`--project-mandatory`
environmentMandatory	This will prevent anyone from using hop-run without specifying an environment	`--environment-mandatory`
defaultProject	The default project to use when none is specified	`--default-project`
defaultEnvironment	The default environment to use when none is specified	`--default-environment`
standardParentProject	The standard parent project to propose when creating new project	`--standard-parent-project`
standardProjectsFolder	The folder to which you’ll browse by default in the GUI when creating new projects	`--standard-projects-folder`

projectMandatory

This will prevent anyone from using hop-run without specifying a project

--project-mandatory

environmentMandatory

This will prevent anyone from using hop-run without specifying an environment

--environment-mandatory

defaultProject

The default project to use when none is specified

--default-project

defaultEnvironment

The default environment to use when none is specified

--default-environment

standardParentProject

The standard parent project to propose when creating new project

--standard-parent-project

standardProjectsFolder

The folder to which you’ll browse by default in the GUI when creating new projects

--standard-projects-folder

Graph database connections

Hop Conf tests graph database connections and Neo4j connections by name, and tells what the graph database types support. The connections come from the metadata of a project: enable it with the -j (project) or -e (environment) option of the hop command, before conf. Without a project, and with hop-conf.sh, the metadata in the folder of the HOP_METADATA_FOLDER variable is used. Variables in the connection settings are resolved with the variables of the project and environment.

Option Description

Option	Description
`--graph-connection-test=<name>`	Connect to the database and print what was tested, for example the URL. The exit code is 0 when the connection works, 1 with the error when it doesn’t or when there is no connection with that name.
`--graph-connection-info=<name>`	Print the name, the metadata type (`graph-database-connection` or `neo4j-connection`) and the graph database type with its capabilities as JSON. Settings like the server or the password are not printed. A Neo4j connection has the type Neo4j.
`--graph-database-types`	Print all installed graph database types with their capabilities as a JSON array. This doesn’t need a project or a connection.

--graph-connection-test=<name>

Connect to the database and print what was tested, for example the URL. The exit code is 0 when the connection works, 1 with the error when it doesn’t or when there is no connection with that name.

--graph-connection-info=<name>

Print the name, the metadata type (graph-database-connection or neo4j-connection) and the graph database type with its capabilities as JSON. Settings like the server or the password are not printed. A Neo4j connection has the type Neo4j.

--graph-database-types

Print all installed graph database types with their capabilities as a JSON array. This doesn’t need a project or a connection.

The same is available in Hop GUI: the Test button and the Supports line of the graph database connection editor.

bash

./hop -j my-project conf --graph-connection-test=memgraph
Graph database connection 'memgraph' is OK: bolt://localhost:7687

./hop -e my-environment conf --graph-connection-info=memgraph
./hop-conf.sh --graph-database-types

The JSON of a graph database type, also used for the type of --graph-connection-info:

json

{
  "pluginId" : "MEMGRAPH",
  "name" : "Memgraph",
  "description" : "Memgraph over the Bolt protocol",
  "documentationUrl" : "https://hop.apache.org/manual/next/metadata-types/graphs/graph-database-connection.html",
  "dialectId" : "MEMGRAPH",
  "queryLanguage" : "Cypher",
  "capabilities" : {
    "cypher" : true,
    "supportingConstraints" : true,
    "supportingIndexes" : true,
    "supportingNodeIndexes" : true,
    "supportingRelationshipIndexes" : true,
    "supportingRelationshipVectorSearch" : true,
    "supportingSchemaChangesInTransactions" : false,
    "supportingSchemaIntrospection" : true,
    "supportingShortestPath" : false,
    "supportingVectorIndexes" : true,
    "supportingVectorSearch" : true
  }
}

queryLanguage is Cypher or Gremlin. The capabilities are listed in Graph Database Connection.

Running Workflows and Pipelines

You can specify an environment or a project when executing a pipeline or a workflow. By doing so you are automatically configuring metadata, variables without too much fuss.

The easiest example is shown by executing the "complex" pipeline from the Apache Beam examples:

Windows
Linux, macOS

shell

hop-run.bat --project samples --file 'beam/pipelines/complex.hpl' --runconfig Direct

Expected output:

shell

C:\<YOUR_PATH>\hop>echo off
===[Environment Settings - hop-run.bat]===================================
Java identified as "C:\Program Files\Microsoft\jdk-11.0.17.8-hotspot\\bin\java"
HOP_OPTIONS="-Xmx2048m" -DHOP_AUDIT_FOLDER=.\audit -DHOP_PLATFORM_OS=Windows
-DHOP_PLATFORM_RUNTIME=Run -DHOP_AUTO_CREATE_CONFIG=Y
Consolidated parameters to pass to HopRun are
--project samples --file beam/pipelines/complex.hpl --runconfig Direct
Command to start HopRun will be:
"C:\Program Files\Microsoft\jdk-11.0.17.8-hotspot\\bin\java" -classpath lib\core\*;lib\beam\*;lib\swt\win64\*
-Djava.library.path=lib\core;lib\beam "-Xmx2048m" -DHOP_AUDIT_FOLDER=.\audit -DHOP_PLATFORM_OS=Windows
-DHOP_PLATFORM_RUNTIME=Run -DHOP_AUTO_CREATE_CONFIG=Y org.apache.hop.run.HopRun  --project samples
--file beam/pipelines/complex.hpl --runconfig Direct
===[Starting HopRun]=========================================================
2022/12/16 14:23:10 - HopRun - Enabling project 'samples'
2022/12/16 14:23:10 - HopRun - Relative path filename specified: config/projects/samples/beam/pipelines/complex.hpl
2022/12/16 14:23:10 - HopRun - Starting pipeline: config/projects/samples/beam/pipelines/complex.hpl
2022/12/16 14:23:21 - General - Created Apache Beam pipeline with name 'complex'
2022/12/16 14:23:21 - General - Handled transform (INPUT) : Customer data
2022/12/16 14:23:21 - General - Handled transform (INPUT) : State data
2022/12/16 14:23:21 - General - Handled Group By (TRANSFORM) : countPerState, gets data from 1 previous transform(s)
2022/12/16 14:23:21 - General - Handled generic transform (TRANSFORM) : uppercase state, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:23:21 - General - Handled Merge Join (TRANSFORM) : Merge join
2022/12/16 14:23:21 - General - Handled generic transform (TRANSFORM) : Lookup count per state, gets data from 1 previous transform(s), targets=0, infos=1
2022/12/16 14:23:21 - General - Handled generic transform (TRANSFORM) : name<n, gets data from 1 previous transform(s), targets=2, infos=0
2022/12/16 14:23:21 - General - Transform Label: N-Z reading from previous transform targeting this one using : name<n - TARGET - Label: N-Z
2022/12/16 14:23:21 - General - Handled generic transform (TRANSFORM) : Label: N-Z, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:23:21 - General - Transform Label: A-M reading from previous transform targeting this one using : name<n - TARGET - Label: A-M
2022/12/16 14:23:21 - General - Handled generic transform (TRANSFORM) : Label: A-M, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:23:21 - General - Handled generic transform (TRANSFORM) : Switch / case, gets data from 2 previous transform(s), targets=4, infos=0
2022/12/16 14:23:21 - General - Transform CA reading from previous transform targeting this one using : Switch / case - TARGET - CA
2022/12/16 14:23:21 - General - Handled generic transform (TRANSFORM) : CA, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:23:21 - General - Transform NY reading from previous transform targeting this one using : Switch / case - TARGET - NY
2022/12/16 14:23:21 - General - Handled generic transform (TRANSFORM) : NY, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:23:21 - General - Transform FL reading from previous transform targeting this one using : Switch / case - TARGET - FL
2022/12/16 14:23:21 - General - Handled generic transform (TRANSFORM) : FL, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:23:21 - General - Transform Default reading from previous transform targeting this one using : Switch / case - TARGET - Default
2022/12/16 14:23:21 - General - Handled generic transform (TRANSFORM) : Default, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:23:21 - General - Handled generic transform (TRANSFORM) : Collect, gets data from 4 previous transform(s), targets=0, infos=0
2022/12/16 14:23:21 - General - Handled transform (OUTPUT) : complex, gets data from Collect
2022/12/16 14:23:21 - General - Executing this pipeline using the Beam Pipeline Engine with run configuration 'Direct'  ----

bash

./sh hop-run.sh --project samples --file 'beam/pipelines/complex.hpl' --runconfig Direct

Expected output:

shell

2022/12/16 14:27:37 - HopRun - Enabling project 'samples'
2022/12/16 14:27:37 - HopRun - Relative path filename specified: config/projects/samples/beam/pipelines/complex.hpl
2022/12/16 14:27:37 - HopRun - Starting pipeline: config/projects/samples/beam/pipelines/complex.hpl
2022/12/16 14:27:41 - General - Created Apache Beam pipeline with name 'complex'
2022/12/16 14:27:41 - General - Handled transform (INPUT) : Customer data
2022/12/16 14:27:41 - General - Handled transform (INPUT) : State data
2022/12/16 14:27:41 - General - Handled Group By (TRANSFORM) : countPerState, gets data from 1 previous transform(s)
2022/12/16 14:27:41 - General - Handled generic transform (TRANSFORM) : uppercase state, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:27:41 - General - Handled Merge Join (TRANSFORM) : Merge join
2022/12/16 14:27:41 - General - Handled generic transform (TRANSFORM) : Lookup count per state, gets data from 1 previous transform(s), targets=0, infos=1
2022/12/16 14:27:41 - General - Handled generic transform (TRANSFORM) : name<n, gets data from 1 previous transform(s), targets=2, infos=0
2022/12/16 14:27:41 - General - Transform Label: N-Z reading from previous transform targeting this one using : name<n - TARGET - Label: N-Z
2022/12/16 14:27:41 - General - Handled generic transform (TRANSFORM) : Label: N-Z, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:27:41 - General - Transform Label: A-M reading from previous transform targeting this one using : name<n - TARGET - Label: A-M
2022/12/16 14:27:41 - General - Handled generic transform (TRANSFORM) : Label: A-M, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:27:41 - General - Handled generic transform (TRANSFORM) : Switch / case, gets data from 2 previous transform(s), targets=4, infos=0
2022/12/16 14:27:41 - General - Transform CA reading from previous transform targeting this one using : Switch / case - TARGET - CA
2022/12/16 14:27:42 - General - Handled generic transform (TRANSFORM) : CA, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:27:42 - General - Transform NY reading from previous transform targeting this one using : Switch / case - TARGET - NY
2022/12/16 14:27:42 - General - Handled generic transform (TRANSFORM) : NY, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:27:42 - General - Transform FL reading from previous transform targeting this one using : Switch / case - TARGET - FL
2022/12/16 14:27:42 - General - Handled generic transform (TRANSFORM) : FL, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:27:42 - General - Transform Default reading from previous transform targeting this one using : Switch / case - TARGET - Default
2022/12/16 14:27:42 - General - Handled generic transform (TRANSFORM) : Default, gets data from 1 previous transform(s), targets=0, infos=0
2022/12/16 14:27:42 - General - Handled generic transform (TRANSFORM) : Collect, gets data from 4 previous transform(s), targets=0, infos=0
2022/12/16 14:27:42 - General - Handled transform (OUTPUT) : complex, gets data from Collect
2022/12/16 14:27:42 - General - Executing this pipeline using the Beam Pipeline Engine with run configuration 'Direct'

To execute an Apache Beam pipeline a lot of information and metadata is needed. Let’s dive into a few fun information tidbits:

By referencing the samples project Hop knows where the project is located (config/projects/samples)
Since we know the location of the project, we can specify pipelines and workflows with a relative path
The project knows where its metadata is stored (config/projects/samples/metadata) so it knows where to find the Direct pipeline run configuration (config/projects/samples/metadata/pipeline-run-configuration/Direct.json)
This run configuration defines its own pipeline engine specific variables, in this case the output folder : DATA_OUTPUT=${PROJECT_HOME}/beam/output/
The output of the samples is as such written to config/projects/samples/beam/output

To reference an environment you can execute using -e or --environment. The only difference is that you’ll have a number of extra environment variables set while executing.

Cloud Storage Configuration

Hop Conf can be used to configure your AWS, Azure and Google Cloud (Cloud Storage and Drive) accounts with Hop through VFS

Amazon Web Services S3

N/A

Azure

Set the account and key of your Azure storage account, or the URL of an Azure emulator (Azurite)

      -aza, --azure-account=<account>
                            The account to use for the Azure VFS
      -aze, --azure-emulator=<emulatorUrl>
                            The URL of an Azure emulator (Azurite) to be used
                              for test and development goals instead of
                              official Azure services
      -azk, --azure-key=<key>
                            The key to use for the Azure VFS

Google

Google Cloud Storage

Set the path to your Google Cloud service account JSON key file, and the block size to read files with

      -gck, --google-cloud-service-account-key-file=<serviceAccountKeyFile>
                            Configure the path to a Google Cloud service account JSON key file
      -gcrbs, --google-cloud-read-block-size=<readBlockSize>
                            Configure the bytes per request when reading a file
                              in blocks, like 8MB

Google Drive

Set the path to your Google Drive credentials JSON file or Google Drive tokens folder.

      -gdc, --google-drive-credentials-file=<credentialsFile>
                            Configure the path to a Google Drive credentials JSON
                              file
      -gdt, --google-drive-tokens-folder=<tokensFolder>
                            Configure the path to a Google Drive tokens folder

Options

The options described on this page. Run hop-conf.sh -h for the complete list of your installation.

Short option Long option Description

Short option	Long option	Description
General
`-h`	`--help`	Displays the help message and quits.
`-v`	`--version`	Prints version information and exits.
`-l`	`--level=<level>`	The log level: NOTHING, ERROR, MINIMAL, BASIC, DETAILED, DEBUG or ROWLEVEL.
Projects
`-p`	`--project=<projectName>`	The name of the project to manage.
`-pc`	`--project-create`	Create a project. Also specify its name and home folder.
`-pm`	`--project-modify`	Modify a project.
`-pd`	`--project-delete`	Remove a project from `hop-config.json`. Its files are not deleted.
`-pl`	`--projects-list`	List the projects.
`-ph`	`--project-home=<projectHome>`	The home folder of the project.
`-pf`	`--project-config-file=<projectConfigFile>`	The configuration file, relative to the home folder. The default is `project-config.json`.
`-pkf`	`--project-keep-config-file`	Keep the configuration file of the project if it already exists.
`-pr`	`--project-parent=<projectParent>`	The parent project to inherit metadata and variables from.
`-ps`	`--project-description=<projectDescription>`	The description of the project.
`-pp`	`--project-company=<projectCompany>`	The company.
`-pt`	`--project-department=<projectDepartment>`	The department.
`-pv`	`--project-variables=<projectVariables>[,…]`	Variables of the project, as `variable=value` separated by commas.
`-pa`	`--project-metadata-base=<projectMetadataBaseFolder>`	The metadata folder, relative to the home folder.
`-pb`	`--project-datasets-base=<projectDataSetsCsvFolder>`	The data sets CSV folder, relative to the home folder.
`-pu`	`--project-unit-tests-base=<projectUnitTestsBasePath>`	The unit tests folder, relative to the home folder.
`-px`	`--project-enforce-execution=<projectEnforceExecutionInHome>`	`true` or `false`: check before execution that a workflow or pipeline is in the home folder of the project or a subfolder.
`-pla`	`--project-list-action-types`	List the action types used in the project.
`-plm`	`--project-list-metadata-types`	List the metadata types used in the project.
`-plt`	`--project-list-transform-types`	List the transform types used in the project.
`-plmi`	`--project-list-metadata-item-usages=<item>`	List the files of the project that use a given metadata item.
`-xm`	`--export-metadata=<metadataJsonFilename>`	Export the metadata of the project given with `-p` to a single JSON file.
Environments
`-e`	`--environment=<environmentName>`	The name of the environment to manage.
`-ec`	`--environment-create`	Create an environment. Also specify its name, purpose, project and configuration files.
`-em`	`--environment-modify`	Modify an environment.
`-ed`	`--environment-delete`	Remove an environment from `hop-config.json`. Its configuration files are not deleted.
`-el`	`--environments-list`	List the environments.
`-ep`	`--environment-project=<environmentProject>`	The project of the environment.
`-eu`	`--environment-purpose=<environmentPurpose>`	The purpose of the environment: Development, Testing, Production, CI, …
`-eg`	`--environment-config-files=<file>[,…]`	The configuration files of the environment, separated by commas.
	`--environment-config-file-directory=<folder>`	A folder of configuration files. Matching files replace the configuration file list, together with `--environment-config-files` when that option is also set.
	`--environment-config-file-wildcard=<regex>`	Regular expression matched against the file names in that folder, for example `.*\.json`.
	`--environment-config-file-exclude-wildcard=<regex>`	Regular expression for file names to ignore in that folder, for example `.secret.`.
	`--environment-config-include-subfolders`	Also include matching files from subfolders of that folder.
Configuration files and variables
`-cfg`	`--config-file=<configFile>`	The configuration file to manage.
`-cfv`	`--config-file-set-variables=<variable=value>[,…]`	Set variables in that file, separated by commas.
`-cfd`	`--config-file-describe-variables=<variable=description>[,…]`	Describe variables in that file, separated by commas.
`-sv`	`--set-variable=<variable=value>`	Set a variable in `hop-config.json`. Without a value, the variable gets an empty value.
`-dv`	`--describe-variable=<variable=description>`	Describe a variable in `hop-config.json`.
Projects plugin
`-pn`	`--projects-enabled`	Enable or disable the projects plugin.
`-py`	`--project-mandatory`	Make it mandatory to reference a project.
`-ey`	`--environment-mandatory`	Make it mandatory to reference an environment.
`-dp`	`--default-project=<defaultProject>`	The default project to use when none is specified.
`-de`	`--default-environment=<defaultEnvironment>`	The default environment to use when none is specified.
`-sp`	`--standard-parent-project=<standardParentProject>`	The parent project proposed when creating a project.
`-sj`	`--standard-projects-folder=<standardProjectsFolder>`	The folder proposed when creating a project.
`-dc`	`--default-projects-folder=<defaultProjectConfigFile>`	The configuration filename for new projects.
Beam and Spark, when the Beam plugin is installed
`-fj`	`--generate-fat-jar=<fatJarFilename>`	Generate a fat jar with the given filename from your current Hop installation, for the Beam and Spark engines.
`-scv`	`--spark-client-version=<sparkClientVersion>`	The Spark client pack to embed in the fat jar: a version (folder `lib/spark-clients/<version>`), `native` or `native-provided`. When omitted, `HOP_SPARK_CLIENT_VERSION` or the default pack in `lib/spark-client` is used.
Graph database connections
	`--graph-connection-test=<name>`	Test the graph database connection or Neo4j connection with the given name. See Graph database connections.
	`--graph-connection-info=<name>`	Print the graph database type and capabilities of a connection as JSON.
	`--graph-database-types`	Print the installed graph database types and their capabilities as JSON.

General

-h

--help

Displays the help message and quits.

-v

--version

Prints version information and exits.

-l

--level=<level>

The log level: NOTHING, ERROR, MINIMAL, BASIC, DETAILED, DEBUG or ROWLEVEL.

Projects

-p

--project=<projectName>

The name of the project to manage.

-pc

--project-create

Create a project. Also specify its name and home folder.

-pm

--project-modify

Modify a project.

-pd

--project-delete

Remove a project from hop-config.json. Its files are not deleted.

-pl

--projects-list

List the projects.

-ph

--project-home=<projectHome>

The home folder of the project.

-pf

--project-config-file=<projectConfigFile>

The configuration file, relative to the home folder. The default is project-config.json.

-pkf

--project-keep-config-file

Keep the configuration file of the project if it already exists.

-pr

--project-parent=<projectParent>

The parent project to inherit metadata and variables from.

-ps

--project-description=<projectDescription>

The description of the project.

-pp

--project-company=<projectCompany>

The company.

-pt

--project-department=<projectDepartment>

The department.

-pv

--project-variables=<projectVariables>[,…]

Variables of the project, as variable=value separated by commas.

-pa

--project-metadata-base=<projectMetadataBaseFolder>

The metadata folder, relative to the home folder.

-pb

--project-datasets-base=<projectDataSetsCsvFolder>

The data sets CSV folder, relative to the home folder.

-pu

--project-unit-tests-base=<projectUnitTestsBasePath>

The unit tests folder, relative to the home folder.

-px

--project-enforce-execution=<projectEnforceExecutionInHome>

true or false: check before execution that a workflow or pipeline is in the home folder of the project or a subfolder.

-pla

--project-list-action-types

List the action types used in the project.

-plm

--project-list-metadata-types

List the metadata types used in the project.

-plt

--project-list-transform-types

List the transform types used in the project.

-plmi

--project-list-metadata-item-usages=<item>

List the files of the project that use a given metadata item.

-xm

--export-metadata=<metadataJsonFilename>

Export the metadata of the project given with -p to a single JSON file.

Environments

-e

--environment=<environmentName>

The name of the environment to manage.

-ec

--environment-create

Create an environment. Also specify its name, purpose, project and configuration files.

-em

--environment-modify

Modify an environment.

-ed

--environment-delete

Remove an environment from hop-config.json. Its configuration files are not deleted.

-el

--environments-list

List the environments.

-ep

--environment-project=<environmentProject>

The project of the environment.

-eu

--environment-purpose=<environmentPurpose>

The purpose of the environment: Development, Testing, Production, CI, …

-eg

--environment-config-files=<file>[,…]

The configuration files of the environment, separated by commas.

--environment-config-file-directory=<folder>

A folder of configuration files. Matching files replace the configuration file list, together with --environment-config-files when that option is also set.

--environment-config-file-wildcard=<regex>

Regular expression matched against the file names in that folder, for example .*\.json.

--environment-config-file-exclude-wildcard=<regex>

Regular expression for file names to ignore in that folder, for example .secret..

--environment-config-include-subfolders

Also include matching files from subfolders of that folder.

Configuration files and variables

-cfg

--config-file=<configFile>

The configuration file to manage.

-cfv

--config-file-set-variables=<variable=value>[,…]

Set variables in that file, separated by commas.

-cfd

--config-file-describe-variables=<variable=description>[,…]

Describe variables in that file, separated by commas.

-sv

--set-variable=<variable=value>

Set a variable in hop-config.json. Without a value, the variable gets an empty value.

-dv

--describe-variable=<variable=description>

Describe a variable in hop-config.json.

Projects plugin

-pn

--projects-enabled

Enable or disable the projects plugin.

-py

--project-mandatory

Make it mandatory to reference a project.

-ey

--environment-mandatory

Make it mandatory to reference an environment.

-dp

--default-project=<defaultProject>

The default project to use when none is specified.

-de

--default-environment=<defaultEnvironment>

The default environment to use when none is specified.

-sp

--standard-parent-project=<standardParentProject>

The parent project proposed when creating a project.

-sj

--standard-projects-folder=<standardProjectsFolder>

The folder proposed when creating a project.

-dc

--default-projects-folder=<defaultProjectConfigFile>

The configuration filename for new projects.

Beam and Spark, when the Beam plugin is installed

-fj

--generate-fat-jar=<fatJarFilename>

Generate a fat jar with the given filename from your current Hop installation, for the Beam and Spark engines.

-scv

--spark-client-version=<sparkClientVersion>

The Spark client pack to embed in the fat jar: a version (folder lib/spark-clients/<version>), native or native-provided. When omitted, HOP_SPARK_CLIENT_VERSION or the default pack in lib/spark-client is used.

Graph database connections

--graph-connection-test=<name>

Test the graph database connection or Neo4j connection with the given name. See Graph database connections.

--graph-connection-info=<name>

Print the graph database type and capabilities of a connection as JSON.

--graph-database-types

Print the installed graph database types and their capabilities as JSON.