Beam Input

Description

The Beam Input transform reads files using a file definition with the Beam execution engine.

Options

Option Description

Transform name

Name of the transform, this name has to be unique in a single pipeline.

Input location

The location from which to read a file.

File definition to use

The file definition to use stored in a metastore.

Edit

Edits an existing file definition.

New

Creates a new file definition.

Manage

Opens the MetaStore Explorer.

File Definition

A file definition can be used to define a file structure definition.

Option Description

Name

The file definition name.

Description

The file definition description.

Field separator

The character separating the fields.

Field enclosure

The character enclosing fields.

Field definitions

A list of fields.

Field name

The field name.

Type

The field data type.

Format

The field format.

Length

The field length.

Precision

The field precisiom.