Row Generator transform Icon Row Generator

Description

To use fields, you must have a data row. A data row will automatically be generated by some transforms like a table input (if you ran the preview). If no data row exists add a “Generate rows” transform to create a row(s) in order to use fields or print variables or parameters etc.

The Generate rows transform outputs a specified number of rows that are empty or have a set of hard-coded fields. By default, the rows are empty; however they can contain a number of static fields.

This transform is used primarily for testing purposes. It may be useful for generating a fixed number of rows, for example, you want exactly 12 rows for 12 months.

Sometimes you may use Generate Rows to generate one row that is an initiating point for your pipeline. For example, you might generate one row that contains two or three field values that you might use to parameterize your SQL and then generate the real rows.

Examples: * To generate a row of sample data, add two action items to a pipeline. Add a Generate rows transform connected to Add constants. In Add constants, you can add fields to generate. In Generate rows, you can limit the number of rows. * To loop through 100 rows you could use a Generate rows and an “Add sequence” Action to the pipeline and connect them on a pipeline.

Supported Engines

Hop Engine

Supported

Spark

Supported

Flink

Supported

Dataflow

Supported

Options

Option Description

Transform Name

Name of the transform this name has to be unique in a single pipeline

Limit

Sets the maximum number of rows you want to generate

Fields

This table is where you configure the structure and values of the rows you are generating (optional).

Never stop generating rows

This option is intended for real-time use cases where you never want to stop the running pipeline. The output of this transform is then used to drive recurring tasks like polling from a file, queue, database, …​

Interval in ms

The interval between generated rows in milliseconds.

Current row time field name

The *optional *field name for the Date field containing the time when the current row was generated

Previous row time field name

The *optional *field name for the Date field containing the time when the previous row was generated