MonetDB Bulk Loader transform Icon MonetDB Bulk Loader

Description

The MonetDB Bulk Loader transform bulk loads data to MonetDB. This significantly speeds up data loading to MonetDB.

Supported Engines

Hop Engine

Supported

Spark

Maybe Supported

Flink

Maybe Supported

Dataflow

Maybe Supported

Options

General

Field Description

Transform name

Specify the unique name of the MongoDB Output transform in the pipeline.

Connection

Select your MonetDB database connection

General Settings tab

This tab contains the destination settings, buffer size and location for the logfile.

Field Description

Target Schema

Specify the database schema that has to be used.

Target Table

Specify the database table, use the Browse button next to this field to use a menu to select the table and schema

Buffer size (rows)

Specify how many rows will be kept in memory before transferring to MonetDB

Log file

Specify the location for the Bulk command logs returned from MonetDB

Truncate table

Remove all data from the destination table before loading the data.

Fully quote all SQL statements

Forces quotes around all objects when executing

MonetDB Settings tab

This tab contains information about the temporary files that are generated to load the data.

Field Description

Field separator

This is the separator that will be used in the Bulk copy command, it is not allowed to have this field in the input data.

Field enclosure

The enclosure character used around values.

Null values represented

Null values will be converted to this string, this allows to differentiate empty strings and null values.

Encoding

File encoding used when generating the files for the copy statement.

Output Fields tab

This tab contains the source to target mapping.

Field Description

Target table field

Field containing the name of the field in the target table

Incoming stream field

Field containing the value we want to insert in target table

Format is ok

Set to Y if the incoming stream’s field is the correct format according to the target datatatype.

NOTE:

This setting is evaluated only when Lazy Conversion is applied.

For example: imagine you are getting values from a text file, your incoming data contains numbers or dates and Lazy Conversion is enabled in the input transform. In this case, the data is not transformed internally to the target data type and is managed as a String by Hop. By setting this flag to Y, we are saying Hop that the incoming data’s value is already in a format clearly understandable by the target database according to the target datatype.