JSON Output

Description

The JSON Output transform allows you to generate JSON blocks based on input transform values.

Output JSON will be available as a javascript array or a javascript object depending on the transform settings.

Supported Engines

Hop Engine

Supported

Single Threaded

Supported

Native Spark

Supported

Beam Spark

Supported

Beam Flink

Supported

Beam Dataflow

Supported

Options

General Tab

General tab allows to specify type of transform operation, output json structure, transform output file. This file will be used to dump all generated json.

Settings Section

Option

Description

Transform name

Name of the transform; this name has to be unique in a single pipeline.

Operation

Specify transform operation type. Currently available 3 types of operation:

Output value - only pass output json as a transform output field, do not dump to output file
Write to file - only write to fie, do not pass to output field
Output value and write to file - dump to file and pass generated json as a transform output file

Json block name

This value will be used as a name for json block.

Nr. rows in a block

Number of rows that are combined as one JSON Array

A new file will be generated for each block

Output value

Output field name

Compatibility mode

Mode to be backwards compatible, more information here

Output File Section

Option	Description
Filename	full path to output file
Append	When checked new rows will be appended to the existing files
Create Parent folder	When checked the parent folders will be created when they do not exist, else the transform will fail if the folder does not exist
Do not open create at start	If not checked: file (and in some cases parent folder) will be created/opened to write during pipeline initialization. If checked: file and parent folder will be created only after transform will get any first input data.
Extension	Output file extension. Default value is 'json'
Encoding	Output file encoding
Include date in filename?	If checked - output file name will contains File name value + current date. This may help to generate unique output files.
Include time in filename	If checked - output file name will contains file creation time. Same as for 'Include date in filename' option
Show filename(s) button	Can be useful to test full output file path
Add file to result filenames?	If checked - created output file path will be accessible form transform result

Option

Description

Filename

full path to output file

Append

When checked new rows will be appended to the existing files

Create Parent folder

When checked the parent folders will be created when they do not exist, else the transform will fail if the folder does not exist

Do not open create at start

If not checked:
file (and in some cases parent folder) will be created/opened to write during pipeline initialization.
If checked:
file and parent folder will be created only after transform will get any first input data.

Extension

Output file extension. Default value is 'json'

Encoding

Output file encoding

Include date in filename?

If checked - output file name will contains File name value + current date. This may help to generate unique output files.

Include time in filename

If checked - output file name will contains file creation time. Same as for 'Include date in filename' option

Show filename(s) button

Can be useful to test full output file path

Add file to result filenames?

If checked - created output file path will be accessible form transform result

Fields Tab

This tab is used to map input transform fields to output json values

Option	Description
Element name	Json element name as a key. For example "A":"B" - A is a element name, B is actual input value mapped for this Element name.
Fieldname	Input transform field name. Use 'Get Fields' button to discover available input fields

Option

Description

Element name

Json element name as a key. For example "A":"B" - A is a element name, B is actual input value mapped for this Element name.

Fieldname

Input transform field name. Use 'Get Fields' button to discover available input fields

Compatibility mode

This part of the documentation will explain the differences when turning on compatibility mode. For all new development this mode is not recommended. Imagine we are generating a simple <key,value> list with keynames "name" and "value" and there corresponding values.

We will be using following settings:

Json block name = "data"
Nr rows in block = 3
Compatibility mode = NOT checked (and this is the default option)

This will output:

First file:

json

{
  "data" : [ {
    "name" : "item 1",
    "value" : "value 1"
  }, {
    "name" : "item 2",
    "value" : "value 2"
  }, {
    "name" : "item 3",
    "value" : "value 3"
  } ]
}

Second file:

json

{
  "data" : [ {
    "name" : "item 4",
    "value" : "value 4"
  } ]
}

If compatibility mode is enabled and the transform has the following settings:

Json block name = "data"
Nr rows in block = 3
'Compatibility mode' is checked

This will output:

First file:

json

{
    "data": [
        {
            "name": "item 1"
        },
        {
            "value": "value 1"
        },
        {
            "name": "item 2"
        },
        {
            "value": "value 2"
        },
        {
            "name": "item 3"
        },
        {
            "value": "value 3"
        }
    ]
}

Second file:

json

{
    "data": [
        {
            "name": "item 4"
        },
        {
            "value": "value 4"
        }
    ]
}

As you can see when turning compatibility mode on, each field will be handles as a separate object.