User Defined Java Class transform Icon User Defined Java Class

Description

The User Defined Java Class transform allows you to enter User Defined Java Class to drive the functionality of a complete transform.

In essence, this transform allows you to program your own plugin in a transform.

The goal of this transform is not to allow a user to do full-scale Java development inside a transform.

Obviously we have a whole plugin system available to help with that part.

The goal is to allow users to define methods and logic with as little as code as possible, executed as fast as possible.

For this we use the Janino project libraries that compile Java code in the form of classes at runtime.

Supported Engines

Hop Engine

Supported

Spark

Maybe Supported

Flink

Maybe Supported

Dataflow

Maybe Supported

Options

Option Description

Transform name

Name of the transform.

Class code

The Java code.

Fields

List of output fields.

  • Fieldname: Output field name.

  • Type: Type of field.

  • Length: Length of the field.

  • Precision: Precision of the field.

Parameters

You can use the Parameters table to avoid using hard-coded string values, such as field names (customer for example).

  • Tag: The parameter tag.

  • Value: The parameter value.

  • Description: Description of the parameter.

Info transforms

Additional transforms to read data from

  • Tag

  • Transform: Which transform to read from.

  • Description

Target transforms

Destination Transform

  • Tag

  • Transform: Which transform to output to.

  • Description

Test class

Tests the class.

Usage

Process rows

The Processor code defines the processRow() method, which is the heart of the transform. This method is called by the pipeline in a tight loop and will continue until false is returned.

String firstnameField;
String lastnameField;
String nameField;

public boolean processRow() throws HopException
{
    // Let's look up parameters only once for performance reason.
    //
    if (first) {
      firstnameField = getParameter("FIRSTNAME_FIELD");
      lastnameField = getParameter("LASTNAME_FIELD");
      nameField = getParameter("NAME_FIELD");
      first=false;
    }

    // First, get a row from the default input hop
    //
    Object[] r = getRow();

    // If the row object is null, we are done processing.
    //
    if (r == null) {
      setOutputDone();
      return false;
    }

    // It is always safest to call createOutputRow() to ensure that your output row's Object[] is large
    // enough to handle any new fields you are creating in this transform.
    //
    Object[] outputRow = createOutputRow(r, data.outputRowMeta.size());

    String firstname = get(Fields.In, firstnameField).getString(r);
    String lastname = get(Fields.In, lastnameField).getString(r);

    // Set the value in the output field
    //
    String name = firstname+" "+lastname;
    get(Fields.Out, nameField).setValue(outputRow, name);

    // putRow will send the row on to the default output hop.
    //
    putRow(data.outputRowMeta, outputRow);

    return true;

Error handling

If you want Hop to handle errors that may occur while running your class in a pipeline, you must implement for your own error handling code. Before adding any error handling code, right-click on the User Defined Java Class transform in the Hop client canvas and select Error Handling in the menu that appears. The resulting transform error handling settings dialog box contains options for specifying an error target transform and associated field names that you will use to implement error handling in your defined code.

try {

Object     numList = strsList.stream()
                        .map( new ToInteger() )
                     .sorted( new ReverseCase() )
                     .collect( Collectors.toList() );

    get( Fields.Out, "reverseOrder" ).setValue( row, numList.toString() );

} catch (NumberFormatException ex) {
    // Number List contains a value that cannot be converteds to an Integer.
    rowInError = true;
    errMsg = ex.getMessage();
    errCnt = errCnt + 1;
}

if ( !rowInError ) {
    putRow( data.outputRowMeta, row );
} else {
    // Output errors to the error hop. Right click on transform and choose "Error Handling..."
    putError(data.outputRowMeta, row, errCnt, errMsg, "Not allowed", "DEC_0");
}

The try in the code sample above tests to see if numList contains valid numbers. If the list contains a number that is not valid, putError is used to handle the error and direct it to the wlog: ErrorPath transform in the sample pipeline. The ErrorPath transform is also specified in the Target transforms tab of the User Define Java Class transform.

Logging

You need to implement logging in your defined transform if you want Hop to log data actions from your class, such as read, write, output, or update data. The following code is an example of how to implement logging:

putRow( data.outputMeta, r );

if ( checkFeedback( getLinesOutput() ) ) {
  if ( log.isBasic() ) {
    logBasic( "Have I got rows for you! " + getLinesOutput() );
  }
}

Class and code fragments

You can navigate through your defined classes along with related code snippets and fields through the Classes and Code Fragments panel. You can right-click on any item in this tree to either Delete, Rename, or Show Sample.

Classes

The Classes folder indicates what classes have corresponding code block tabs in the Class Code panel.

Code Snippits

The Code Snippits folder shows the internal Hop code related to the User Defined Java Class transform. These snippits are shown as reference for the code of your class.

Input Fields

The Input fields folder contains any input fields you define in your code. While working with your defined code, you will be handling input and output fields. Many ways exist for handling input fields. For example, to start, examine the following description of an input row.

IRowMeta inputRowMeta = getInputRowMeta();

The inputRowMeta object contains the metadata of the input row. It includes all the fields, their data types, lengths, names, format masks, and more. You can use this object to look up input fields. For example, if you want to look for a field called customer, you would use the following code.

IValueMeta customer = inputRowMeta.searchValueMeta("year");

Because looking up field names can be slow if you need to do it for every row that passes through a pipeline, you could look up field names in advance in a first block of code, as shown in the following example:

if (first) {
 yearIndex = getInputRowMeta().indexOfValue(getParameter("YEAR"));
 if (yearIndex<0) {
   throw new HopException("Year field not found in the input row, check parameter 'YEAR'\!");
 }
}

To get the Integer value contained in the year field, you can then use the following construct.

Object[] r = getRow();
...
Long year = inputRowMeta().getInteger(r, yearIndex);

To make this process easier, you can use a shortcut in the following form.

Long year = get(Fields.In, "year").getInteger(r);

This method also takes into account the index-based optimization mentioned above.

Blocking specific code

As a simple security measure you can block the execution of code containing specific strings. This can be done by adding exclusions to the codeExclusions.xml file located at <Hop Installation>/plugins/transforms/janino

Example:

    <exclusions>
        <exclusion>System.</exclusion>
        <exclusion>HopVfs.</exclusion>
    </exclusions>