MonetDB Bulk Loader

Description

The MonetDB Bulk Loader transform bulk loads data to MonetDB. This significantly speeds up data loading to MonetDB.

Options

General

Transform name : Specify the unique name of the MongoDB Output transform in the pipeline. Connection : Select your MonetDB database connection

General Settings tab

This tab contains the destination settings, buffer size and location for the logfile.

Field	Description
Target Schema	Specify the database schema that has to be used.
Target Table	Specify the database table, use the Browse button next to this field to use a menu to select the table and schema
Buffer size (rows)	Specify how many rows will be kept in memory before transferring to MonetDB
Log file	Specify the location for the Bulk command logs returned from MonetDB
Truncate table	Remove all data from the destination table before loading the data.
Fully quote all SQL statements	Forces quotes around all objects when executing

Field

Description

Target Schema

Specify the database schema that has to be used.

Target Table

Specify the database table, use the Browse button next to this field to use a menu to select the table and schema

Buffer size (rows)

Specify how many rows will be kept in memory before transferring to MonetDB

Log file

Specify the location for the Bulk command logs returned from MonetDB

Truncate table

Remove all data from the destination table before loading the data.

Fully quote all SQL statements

Forces quotes around all objects when executing

MonetDB Settings tab

This tab contains information about the temporary files that are generated to load the data.

Field	Description
Field separator	This is the separator that will be used in the Bulk copy command, it is not allowed to have this field in the input data.
Field enclosure	The enclosure character used around values.
Null values represented	Null values will be converted to this string, this allows to differentiate empty strings and null values.
Encoding	File encoding used when generating the files for the copy statement.

Field

Description

Field separator

This is the separator that will be used in the Bulk copy command, it is not allowed to have this field in the input data.

Field enclosure

The enclosure character used around values.

Null values represented

Null values will be converted to this string, this allows to differentiate empty strings and null values.

Encoding

File encoding used when generating the files for the copy statement.

Output Fields tab

This tab contains the source to target mapping.

Field	Description
Target table field	Field containing the name of the field in the target table
Incoming stream field	Field containing the value we want to insert in target table
Format is ok	Set to Y if the incoming stream’s field is the correct format according to the target datatatype. NOTE: This setting is evaluated only when Lazy Conversion is applied. For example: imagine you are getting values from a text file, your incoming data contains numbers or dates and Lazy Conversion is enabled in the input transform. In this case, the data is not transformed internally to the target data type and is managed as a String by Hop. By setting this flag to Y, we are saying Hop that the incoming data’s value is already in a format clearly understandable by the target database according to the target datatype.

Field

Description

Target table field

Field containing the name of the field in the target table

Incoming stream field

Field containing the value we want to insert in target table

Format is ok

Set to Y if the incoming stream’s field is the correct format according to the target datatatype.

NOTE:

This setting is evaluated only when Lazy Conversion is applied.

For example: imagine you are getting values from a text file, your incoming data contains numbers or dates and Lazy Conversion is enabled in the input transform. In this case, the data is not transformed internally to the target data type and is managed as a String by Hop. By setting this flag to Y, we are saying Hop that the incoming data’s value is already in a format clearly understandable by the target database according to the target datatype.