Avro encode

Description

The Avro encode transform allows you to encode a new Avro Record field using a selection of Hop fields. The Avro schema will be part of the value metadata for this Avro Record field.

Avro encode does not run on the native Spark engine. The native Spark engine has no Spark data type to carry the Avro Record field this transform produces from one transform to the next, so the pipeline is refused before it starts. To write Avro on the native Spark engine, use the Avro file output transform.

Options

Option	Description
Transform name	Name of the transform. Note: This name has to be unique in a single pipeline.
Output field name	Choose a name for the Avro Record output field.
Schema name	The name of the schema included in the output Avro record
Namespace	An optional schema namespace
Documentation	An optional schema documentation (doc) element
The fields to encode in a generic Avro record:	You can specify the names of the Hop input fields to be included in the output Avro record field. Optionally you can store the values under a different name (key) in the Avro record.
Get fields button	You can use the "Get fields" button to retrieve the fields to be included in the Avro record.

Option

Description

Transform name

Name of the transform. Note: This name has to be unique in a single pipeline.

Output field name

Choose a name for the Avro Record output field.

Schema name

The name of the schema included in the output Avro record

Namespace

An optional schema namespace

Documentation

An optional schema documentation (doc) element

The fields to encode in a generic Avro record:

You can specify the names of the Hop input fields to be included in the output Avro record field. Optionally you can store the values under a different name (key) in the Avro record.

Get fields button

You can use the "Get fields" button to retrieve the fields to be included in the Avro record.