Simple Mapping transform Icon Simple Mapping

Description

The Simple Mapping transform allows you to re-use a series of transforms in the form of a "re-usable pipeline" also known as a "Mapping".

In the Simple Mapping transform you can specify one Mapping Input transform to pass data to and one Mapping Output transform to retrieve rows of data from.

Use this transform if you find yourself repeating the same logic multiple times over several different pipelines.

Supported Engines

Hop Engine

Supported

Spark

Supported

Flink

Supported

Dataflow

Supported

Options

The options are fairly self-explanatory:

  • Transform name: a unique name in your pipeline

  • Pipeline: The name of the mapping (sub-) pipeline to embed. This pipeline is executed separately and is considered a separate entity. It is always executed with a standard local Hop pipeline run configuration since that is currently the only one capable of streaming data into and out of transforms.

  • Parameters tab: specify the parameters you want to pass to the mapping pipeline

  • Input tab: specify which fields to pass to the Mapping Input transform in the mapping. You can map (hence the name) field names.

  • Output tab: specify how to rename output fields retrieved from the Mapping Output transform.

Sample

In the samples project you can find a small example of the Simple Mapping transform called `mapping-use.hpl'.