Row Generator transform Icon Row Generator

Description

To use fields, you must have a data row. A data row will automatically be generated by some transforms like a table input (if you ran the preview). If no data row exists add a “Generate rows” transform to create a row(s) in order to use fields or print variables or parameters etc.

The Generate rows transform outputs a specified number of rows that are empty or have a set of hard-coded fields. By default, the rows are empty; however they can contain a number of static fields.

This transform is used primarily for testing purposes. It may be useful for generating a fixed number of rows, for example, you want exactly 12 rows for 12 months.

Sometimes you may use Generate Rows to generate one row that is an initiating point for your pipeline. For example, you might generate one row that contains two or three field values that you might use to parameterize your SQL and then generate the real rows.

Examples: * To generate a row of sample data, add two action items to a pipeline. Add a Generate rows transform connected to Add constants. In Add constants, you can add fields to generate. In Generate rows, you can limit the number of rows. * To loop through 100 rows you could use a Generate rows and an “Add sequence” Action to the pipeline and connect them on a pipeline.

Supported Engines

Hop Engine

Supported

Single Threaded

Supported

Native Spark

Supported

Beam Spark

Supported

Beam Flink

Supported

Beam Dataflow

Supported

Options

Option Description

Transform Name

Name of the transform this name has to be unique in a single pipeline

Limit

Sets the maximum number of rows you want to generate

Fields

Structure and constant values of the generated rows (optional). See Fields.

Never stop generating rows

This option is intended for real-time use cases where you never want to stop the running pipeline. The output of this transform is then used to drive recurring tasks like polling from a file, queue, database, …​

Interval in ms

The interval between generated rows in milliseconds.

Current row time field name

The *optional *field name for the Date field containing the time when the current row was generated

Previous row time field name

The *optional *field name for the Date field containing the time when the previous row was generated

Fields

Each row in the Fields table defines one output field. Every generated pipeline row carries the same constants.

The columns below appear on many field tables. They are field metadata: they control how Hop converts a string to a typed value, and how a typed value is written back to a string. They do not change the value stored in a database.

Column Description

Format / Conversion mask

Number or date pattern used to parse and print the value. See Formatting numbers and dates.

Length

String: maximum characters. Number / BigNumber: significant digits. Integer: display width (leading zeros when writing). Date: length of the printed string (for example 4 prints only the year). Use -1 when unset.

Precision

Decimal digits after the radix for Number and BigNumber. Not used for String, Date or Boolean.

Decimal

Character that separates the integer and fractional parts. . in 10,000.00 (US) or , in 10.000,00 (DE).

Group / Grouping

Thousands grouping symbol. , in 10,000.00 (US) or . in 10.000,00 (DE). This is not a row group or category.

Currency

Optional currency symbol used when the format mask contains a currency sign (¤). Examples: $, €.

Value

A single constant, entered as a string and parsed with Type, Format, Decimal, Group and Currency. Not a comma-separated list of values. Use Data Grid when you need several different rows.

Set empty string?

When the constant is empty, emit an empty string ("") instead of null.
Column Description

Name

Name of the generated field.

Type

Hop data type. The Value string is parsed into this type. See Data types.
Value is a single constant, not a comma-separated list. 1,2,3 is one string (or a parse error), not three rows. Group is the thousands separator (, in 1,000.00), not a row group.

To emit several different rows of sample data, use Data Grid instead.