Pipeline Probe icon Pipeline Data Probe

Description

The Pipeline Data Probe transform receives data probe data configured by a Pipeline Probe metadata type.

Check the Pipeline Probe metadata type for more information.

Supported Engines

Hop Engine

Supported

Spark

Maybe Supported

Flink

Maybe Supported

Dataflow

Maybe Supported

Options

The pipeline data probe transform doesn’t require any configuration except for the transform name.

This transform receives data from the Pipeline Probe metadata type and produces the following output:

sourcePipelineName the name of the pipeline passing data to this pipeline data probe transform.

sourceTransformLogChannelId

the id for the log channel passing data to this pipeline data probe transform.

sourceTransformName

the name of the transform in the source pipeline passing data to this pipeline data probe transform.

sourceTransformCopy

the copy number of the transform in the source pipeline passing data to this pipeline data probe transform.

rowNr

the row number in the source pipeline passing data to this pipeline data probe transform.

fieldName

the field name of the field in the source pipeline passing data to this pipeline data probe transform.

fieldType

the field type of the field in the source pipeline passing data to this pipeline data probe transform.

fieldFormat

the field format of the field in the source pipeline passing data to this pipeline data probe transform.

fieldLength

the field length of the field in the source pipeline passing data to this pipeline data probe transform.

fieldPrecision

the field precision of the field in the source pipeline passing data to this pipeline data probe transform.

value

the value of the transform in the source pipeline passing data to this pipeline data probe transform.