Beam Output

Description

The Beam Output transform writes files using a file definition with the Beam execution engine.

Options

Option Description

Transform name

Name of the transform, this name has to be unique in a single pipeline.

Output location

The output location.

File prefix

Text which has to precede the file name.

File suffix

Text which has to succeed the file name.

Windowed writes?

Writes a file per window. Use together with the Beam Window transform.

File definition to use

The file definition to use stored in a metastore.

Edit

Edits an existing file definition.

New

Creates a new file definition.

Manage

Opens the MetaStore Explorer.

File Definition

A file definition can be used to define a file structure definition.

Option Description

Name

The file definition name.

Description

The file definition description.

Field separator

The character separating the fields.

Field enclosure

The character enclosing fields.

Field definitions

A list of fields.

Field name

The field name.

Type

The field data type.

Format

The field format.

Length

The field length.

Precision

The field precisiom.