Token Replacement

Description

The Token Replacement transform replaces tokens in an input string or file.

The transform can then output this data either to a file or a field on the stream.

A token contains a start string, a name, and an end string.

For example ${my_token} could be a token.

The start string, and end string are configurable and can be any series of characters.

When replacing tokens in a file, this transform reads the entire file. It does not operate on a single line at a time. When replacing tokens in a file it is a best practice to output to a file also to prevent Hop from having to read the entire file into memory.

Options

Input Tab

  • Input Type - Where to read the field to do the token replacement from. Either text, field, or file.

  • Input Type Text

    • Input Text - The text to token replace

  • Input Type Field

    • Input Field - The input field to token replace

  • Input Type File

    • Input filename - The name of the file to token replace

    • Filename is in field?

      • Is the name of the file to token replace in a field?

    • Input filename field - The field the name of the file is in.

    • Add Input filename to result?

      • Add the input filename(s) to the result files list.

Output Tab

  • Output Type - Where to put the token replaced string. Either field or file.

  • Output Type Field

    • Output field name - The name of the field to put the token replaced string in.

  • Output Type File

    • Output filename - The name of the file to write to.

    • Filename is in field?

      • Is the name of the output file in a field?

    • Output filename field - The name of the field the output filename is in.

    • Append output file?

      • If the output file already exists should it be appended to. If not checked, Pentaho will overwrite the file if it exists.

    • Create parent folder?

      • Should Pentaho create the parent folder?

    • Output format - The format of the new line delimiter for the output file.

    • Output encoding - The character encoding to use when writing the file.

    • Split every - Split the output file into a new file every n rows.

    • Include stepnr in filename?

      • Should the step number be included in the output filename?

    • Include partition nr in filename?

      • Should the partition number be included in the output filename?

    • Include date in filename?

      • Should the curren date be included in the output filename?

    • Include time in filename?

      • Should the current time be included in the output filename?

    • Specify date format?

      • Do you want to specify the date format to include in the output filename?

    • Date time format - The date/time format to include in the output filename.

    • Add output filenames to result?

      • Add the output filename(s) to the result files list.

Tokens Tab

  • Token start string - The string that indicates the start of a token.

  • Token end string - The string that indicates the end of a token. Everything between the token start string and the token end string is the token name.

  • Stream name - The name of the field on the stream containing the value to replace the token with.

  • Token name - The name of the token to replace.

  • Get Fields button - Gets the list of input fields, and tries to map them to an Avro field by an exact name match.