Set Variables transform Icon Set Variables

Description

Set Variables allows you to set variables from fields. By clicking the Get Fields button it will automatically fill in all the fields from the data row and fill in variable names in all-caps. Verify the scope is correct.

Supported Engines

Hop Engine

Supported

Spark

Maybe Supported

Flink

Maybe Supported

Dataflow

Maybe Supported

This transform accepts one (and only one) row of data to set the value of a variable. It is important to isolate a single row for this transform, a simple way to do this is using the Blocking transform

Variables have no type in Hop (casted as strings). If working with formulas for example, it may not have a type so be sure to convert to string if necessary (can use the TEXT function).

Variables set with Set Variable are not available to use in the current pipeline. If set values are needed in the current pipeline the values will have to come from fields.

Note that variables cannot be passed upstream between pipelines. Parameters are best passed downstream to avoid threading issues. A nested pipeline is technically the same pipeline, so variables are inherited in the initialization phase.

Though you cannot pass parameters and variables upstream (in nested or sequential pipelines) you can pass data rows back up a pipeline via the following pattern in the samples project: samples/loops/pipeline-executor.hpl

A variable can be set in one pipeline and be available in the next pipeline (named pipeline) that is in the loop of a pipeline executor. If you are using a pipeline executor child, the parent pipeline does not restart and does not get any set variables. The new variable name to set in a child pipeline is shown below in the second column.

This transform accepts one (and only one) row of data to set the value of a variable. If you want to set a variable from a pipeline that processes multiple lines of data, make sure you isolate a single row before setting a variable.
You can’t set and use a variable in the same pipeline. Since all transforms in a pipeline run in parallel, there is no way a transform in a pipeline can use a variable that is set in another transform in that same pipeline.

Options

Option Description

Transform name

Name of the transform.

Apply formatting

When this option is set, it formats the value (date, numbers etc.) according to the format options. Otherwise no formatting is done.

Field Name

Name of the field that is used

Variable Name

Name of the variable to set (without ${…​} or %%…​%% sigils)

Variable scope type

Define the scope of the variable, possible options are:

  • Valid in the Java Virtual Machine: the complete virtual machine will know about this variable.

  • Valid in the parent workflow: the variable is only valid in the parent workflow.

  • Valid in the grand-parent workflow: the variable is valid in the grand-parent workflow and all the child workflows and pipelines.

  • Valid in the root workflow: the variable is valid in the root workflow and all the child workflows and pipelines.

Default value

Value that will be set for empty rows