JavaScript transform Icon JavaScript

Description

The JavaScript transform provides a user interface for building JavaScript expressions that you can use to modify your data. The code you type in the script area is executed once for each input row.

Variables won’t be usable or testable until you create fields. Click the “Get variables” button to convert script variables into fields.

Supported Engines

Hop Engine

Supported

Spark

Maybe Supported

Flink

Maybe Supported

Dataflow

Maybe Supported

Usage

The JavaScript transform provides a user interface for building JavaScript expressions that you can use to modify your data. The code you type in the script area is executed once for each input row.

The transform allows multiple scripts in a single transform instance.

The Javascript transform is not an input transform and therefore requires an input stream from the pipeline.

Minimizing scripting is very important to keep your data integration solutions maintainable. Try to avoid using JavaScript as much as possible.

Javascript Transform Dialog

Troubleshooting

  • Hop uses an older version of JavaScript (Rhino engine) which is ECMA 5 and a bit of ECMA 6. See Rhino ES2015/ES6, ES2016 and ES2017 support at https://mozilla.github.io/rhino/compat/engines.html

  • The Get variables button may not always work, so enter variables manually.

  • The JavaScript transform runs better with less whitespace and less line continuations, especially when using GraphQL. Avoid multi-line expressions or statements.

  • Ensure you select the correct Type for the outgoing JavaScript fields.

Examples

E.g. JavasScript to create 4 new fields:

var myVar = incomingFieldFromHop;
var myTestString = "my test";
var myDate = str2date("2020-12-31", "yyyy-MM-dd");
var myDateTime = new Date("2023-10-01T01:40:26.210");

E.g. JavaScript to flatten JSON keys:

//var input_json = {
//    "c-102": "AIDS Healthcare",
//    "c-105": "AIDS Healthcare Direct",
//    "c-75": "Allied Physicians (ALIP)",
//    "c-59": "Asheville Endocrinology"};

var input_json = JSON.parse(incomingJSONFromHop);
var output_json = [];

for (var key in input_json) {
    var value = input_json[key];
    output_json.push({field1: key, field2: value });
}

var flattened_json = JSON.stringify(output_json);

Javascript functions Panel

Javascript Functions Panel

The Javascript functions panel contains a tree view of scripts, constants, functions, input fields, and output fields as described below.

Double click on any of the scripts, constants, functions or fields to add them to the script.

Transform Scripts

Scripts you have created in this transform.

Transform Constants

A number of pre-defined, static constants that control what happens to the data rows.
To use these constants, you must first set a pipeline_Status variable to CONTINUE_PIPELINE at the beginning of the script, so that the variable assignment is made to the first row being processed. Otherwise, any subsequent assignments to the pipeline_Status variable are ignored.
The available constants are:

  • SKIP_PIPELINE: Excludes the current row from the output row set and continues processing on the next row.

  • ERROR_PIPELINE: Excludes the current row from the output row set, generates an error, and any remaining rows are not processed.

  • CONTINUE_PIPELINE: Includes the current row in the output row set.

  • ABORT_PIPELINE: Excludes the current row from the output row set, and any remaining rows are not processed, but does not generate an error. (This constant does not display in the transform dialog, but can be used in your script)

Transform Functions

String, numeric, date, logic, special, and file functions you can use in scripts. These included functions are implemented in Java and execute faster than JavaScript functions. Each function has a sample script demonstrating its use. Double-click the function to add it to the Javascript pane. Right-click and choose Sample to add the sample to the Javascript pane.

Input Fields

Input fields for the transform.

Output Fields

Output fields for the transform.

Javascript Panel

The Javascript pane is the editing area for writing your code. You can insert constants, functions, input fields, and output fields from the Javascript functions panel on the left by double-clicking the node you want to insert or by using drag-and-drop to place the object onto the Javascript panel.

The position at the bottom of the Javascript panel displays the line number and position of the cursor.

The Optimization level selects the level of JavaScript optimization. The values are:

  • 1: JavaScript runs in interpreted mode.

  • 0: No optimizations are performed.

  • 1-9: All optimizations are performed. 9 performs the most optimization with faster script execution, but compiles slower. The default is 9.

Script types

Javascript transform right click menu

You can right-click a tab in the Javascript panel to open a context menu with the following commands:

  • Add new – Add a new script tab.

  • Add copy – Add a copy of the existing script in a new tab.

  • Set Transform Script - Specify the script to execute for each incoming row. Only one tab can be set as a transform script. The first tab is a transform script by default.

  • Set Start Script - Specify the script to execute before processing the first row.

  • Set End Script – Specify the script to execute after the last row has been processed.

  • Remove Script Type - Specify to not execute the script. The script tab is not removed. To remove a script tab, click the Close button (the red “X”) and choose Yes to delete the script tab.

The icon for the script type displays on the tab to denote the type of script on the tab. To rename a script tab, right-click the tab name in the Transform Scripts section of the Javascript functions panel, choose Rename, and enter the new name.

Fields table

The Fields table contains a list of variables from your script, and enables you to add metadata to the fields, like descriptive names.

Field Description

Fieldname

Specify the name of the incoming field.

Rename to

Specify a new name for the incoming field.

Type

Specify a data type for the output field.

Length

Specify the length of the output field.

Precision

Specify the precision value of the output field.

Replace value ‘Fieldname’ or ‘Rename to’

Specify whether to replace the value of the selected field with another value or to rename a field. The values are Y (Yes) and N (No).

Get variables

Retrieve a list of Javascript variables from your script. You can manually add variables as the ‘Get variables’ button may not always work.

Test Script

Test the syntax of your script, and displays the Generate Rows dialog box with a set of rows for testing.

Javascript Internal API Objects

You can use the following internal API objects (for reference see the classes in the source):

  • PipelineName: A String holding the pipeline name

  • transform: the actual transforms instance for this transform (org.apache.hop.pipeline.transforms.javascript.ScriptValues)

  • rowMeta: The actual instance of org.apache.hop.core.row.IRowMeta

  • row: The actual instance of the data Object[]

Examples

Check for the Existence of Fields in a row:

var idx = getInputRowMeta().indexOfValue("lookup");
if ( idx < 0 )
{
   var lookupValue = 0;
}
else
{
   var lookupValue = row[idx];
}

Add a New Field in a Row

Fields must be added to the rows in the same order to keep the structure of the row coherent.

To add a field, define it as var in the Javascript pane, and add it as a field in the Fields table.

Numeric values

Most values that are assigned in JavaScript are floating point values by default, even if you think you have assigned an integer value. If you are having trouble using == or switch/case on values that you know are integers, use the following constructs:

switch(parseInt(valuename))
{
case 1:
case 2:
case 3:
 strvalueswitch = "one, two, three";
 break;
case 4:
 strvalueswitch = "four";
 break;
default:
 strvalueswitch = "five";
}

Filter Rows

To filter rows (remove the rows from the output for example) set the pipeline_Status variable as follows:

pipeline_Status = CONTINUE_PIPELINE
if (/* your condition here */) pipeline_Status = SKIP_PIPELINE

All rows matching the specified condition are removed from the output.

Use field names with spaces

Javascript does not allow you to create variables with spaces in the variable name. However, you can work with field names that contain spaces.

For example, to replace all spaces with underscores in a field field name with spaces, use the syntax this["field name with spaces"].

The this keyword is required. Without it, Javascript will use field name with spaces as a character array.

var new_field = replace(this["field name with spaces"], " ", "_");

Produce a custom JSON including nested arrays

You can create your own custom json structure with a javascript similar to this one. For this example you need to create two input fields short_filename and myField1 i.e. by using a data grid transform. The resulting JSON "myBodyJson" can then be used i.e. in a REST Client transform. For a POST request just map this "myBodyJson" to "Body field".

var json = {};
json.sourceProperties = {};
json.sourceProperties.properties = [];

json.filename = short_filename;
json.sourceCategory = "CategoryABC";

json.sourceProperties.properties.push({"key": "exampleArr", "values": [1000]})
json.sourceProperties.properties.push({"key": "exampleString", "values": "Some Testingtext"})
json.sourceProperties.properties.push({"key": "exampleUsingInputField", "values": myField1})

var myBodyJson = JSON.stringify(json);