Kettle Import

Whenever you want to import metadata from Kettle (Pentaho Data Integration) you can use this plugin.

Usage

You can use it through the File / Import from Kettle/PDI menu in the Hop GUI or using the hop-import script with the --type kettle option.

Functionality

In general the plugin copies Kettle files and non-Kettle files.

Kettle files

File Conversion

File	Conversion
Transformations	Kettle transformation files (.ktr extension) are converted to Hop pipelines. During this process the XML metadata is converted to the appropriate Hop format.
Jobs	Kettle job files (.kjb extension) are converted to Hop workflows. During this process the XML metadata is converted to the appropriate Hop format.
kettle.properties	The Kettle properties file in your .kettle folder (typically found in the home directory or `${KETTLE_HOME}`) often contains variables and values regarding your environment. These variables and values are converted into an environment configuration file if you specified the `-c` or `--target-config-file` option. When you create an environment in Hop you can simply add this file to it to make everything work. If the configuration file already exists it will be updated, not overwritten. The description of the newly imported variables is set to `Imported from Kettle` to indicate that it’s new. Values of existing variables are overwritten and the existing description is kept.
shared.xml	The shared.xml file in your .kettle folder (typically found in the home directory or `${KETTLE_HOME}`) often contains connections which are shared across many transformations and jobs. These connections are imported as Relational Database Connection metadata stored in the target folder `metadata/rdbms` folder.
jdbc.properties	A jdbc.properties file sometimes contains relational database metadata which is converted into generic RDBMS connections metadata.

Transformations

Kettle transformation files (.ktr extension) are converted to Hop pipelines. During this process the XML metadata is converted to the appropriate Hop format.

Jobs

Kettle job files (.kjb extension) are converted to Hop workflows. During this process the XML metadata is converted to the appropriate Hop format.

kettle.properties

The Kettle properties file in your .kettle folder (typically found in the home directory or ${KETTLE_HOME}) often contains variables and values regarding your environment. These variables and values are converted into an environment configuration file if you specified the -c or --target-config-file option. When you create an environment in Hop you can simply add this file to it to make everything work. If the configuration file already exists it will be updated, not overwritten. The description of the newly imported variables is set to Imported from Kettle to indicate that it’s new. Values of existing variables are overwritten and the existing description is kept.

shared.xml

The shared.xml file in your .kettle folder (typically found in the home directory or ${KETTLE_HOME}) often contains connections which are shared across many transformations and jobs. These connections are imported as Relational Database Connection metadata stored in the target folder metadata/rdbms folder.

jdbc.properties

A jdbc.properties file sometimes contains relational database metadata which is converted into generic RDBMS connections metadata.

Non-Kettle files

Non-Kettle files like text files, JSON, XML, … they are all copied over by default, including the sub-folders. You can change this behavior with the options:

--skip-folders : don’t copy sub-folders from the input folder
--skip-hidden : Don’t copy hidden files and folders like .git, .gitignore and so on
--skip-existing : Only copy files which haven’t been copied yet

Import Form Options

When using the Import from Kettle form in the Hop GUI, the following options are available:

Option Description Default Value

Option	Description	Default Value
Import from	Select the source Kettle directory or project.	-
Import in existing project	Choose whether to import into an already existing Hop project.	Selected
Import in project	Specify the target project for the imported data.	-
Import to folder	Define the folder path where imported files will be stored.	- To enable this option, deselect the Import in existing project option
Path to `kettle.properties`	Specify the path to the `kettle.properties` file if it’s not in the default location.	-
Path to `shared.xml`	Indicate the path to the `shared.xml` file if necessary.	-
Path to `jdbc.properties`	Provide the path to the `jdbc.properties` file if needed.	-
Skip existing target files?	Enable this option to skip copying files that are already present in the target directory.	Selected
Skip hidden files and folders?	Choose to exclude hidden files and folders during the import.	Selected
Skip folders in the source?	Opt to exclude sub-folders from being imported.	Selected
Target environment variables config file	Specify the name of the environment configuration file where variables from `kettle.properties` will be stored.	`imported-env-conf.json`
Pipeline default run configuration	Set the default run configuration for imported pipelines.	-
Workflow default run configuration	Set the default run configuration for imported workflows.	-

Import from

Select the source Kettle directory or project.

Import in existing project

Choose whether to import into an already existing Hop project.

Selected

Import in project

Specify the target project for the imported data.

Import to folder

Define the folder path where imported files will be stored.

To enable this option, deselect the Import in existing project option

Path to kettle.properties

Specify the path to the kettle.properties file if it’s not in the default location.

Path to shared.xml

Indicate the path to the shared.xml file if necessary.

Path to jdbc.properties

Provide the path to the jdbc.properties file if needed.

Skip existing target files?

Enable this option to skip copying files that are already present in the target directory.

Selected

Skip hidden files and folders?

Choose to exclude hidden files and folders during the import.

Selected

Skip folders in the source?

Opt to exclude sub-folders from being imported.

Selected

Target environment variables config file

Specify the name of the environment configuration file where variables from kettle.properties will be stored.

imported-env-conf.json

Pipeline default run configuration

Set the default run configuration for imported pipelines.

Workflow default run configuration

Set the default run configuration for imported workflows.