SFTP Put

Description

The SFTP Put transform uploads a file, or the content of a field, to an SFTP server. It does so once per incoming row, so the file to upload, the folder to upload it to and the name it gets on the server all come from fields.

Where that server is and how to log in to it comes from an SFTP Connection in your project metadata, not from the transform itself.

Supported Engines

Hop Engine

Supported

Single Threaded

Supported

Native Spark

Maybe Supported

Beam Spark

Maybe Supported

Beam Flink

Maybe Supported

Beam Dataflow

Maybe Supported

Options

Option Description

Option	Description
Transform name	The name of the transform, unique in a pipeline.
SFTP connection	The SFTP Connection to upload through. Use Edit to change it or New to create one.
Source file/content field	The field holding the name of the file to upload. With The field holds the content enabled, the field holds the data to upload instead.
The field holds the content, not a file name	Upload the content of the source field rather than a file it points at. The remote file name field is required in that case, because there’s no file to take a name from.
Remote folder field	The field holding the folder on the server to upload to. An absolute path on the server, unless the connection has Paths are relative to the home directory enabled.
Remote file name field	The field holding the name the file gets on the server. When it’s empty, the name of the source file is used.
Create the remote folder if it doesn’t exist	Create the remote folder, including its parents. Without this, a missing folder is an error.
After the upload	What to do with the source file once it’s on the server: * `Do nothing` : leave it where it is * `Delete the source file` : remove it * `Move the source file` : move it to the folder in the Move to folder field
Move to folder field	The field holding the folder to move the source file to. Only used with `Move the source file`.
Create the destination folder if it doesn’t exist	Create the folder to move the source file to.
Add the source file name to the result files	Add the file to the result file list of the pipeline, so a later action can pick it up. With `Delete the source file` nothing is added, since there’s no file left to point at.

Transform name

The name of the transform, unique in a pipeline.

SFTP connection

The SFTP Connection to upload through. Use Edit to change it or New to create one.

Source file/content field

The field holding the name of the file to upload. With The field holds the content enabled, the field holds the data to upload instead.

The field holds the content, not a file name

Upload the content of the source field rather than a file it points at. The remote file name field is required in that case, because there’s no file to take a name from.

Remote folder field

The field holding the folder on the server to upload to. An absolute path on the server, unless the connection has Paths are relative to the home directory enabled.

Remote file name field

The field holding the name the file gets on the server. When it’s empty, the name of the source file is used.

Create the remote folder if it doesn’t exist

Create the remote folder, including its parents. Without this, a missing folder is an error.

After the upload

What to do with the source file once it’s on the server:

* Do nothing : leave it where it is * Delete the source file : remove it * Move the source file : move it to the folder in the Move to folder field

Move to folder field

The field holding the folder to move the source file to. Only used with Move the source file.

Create the destination folder if it doesn’t exist

Create the folder to move the source file to.

Add the source file name to the result files

Add the file to the result file list of the pipeline, so a later action can pick it up. With Delete the source file nothing is added, since there’s no file left to point at.

Error handling

The transform supports error handling: enable it on the hop leaving the transform to send the rows which failed to upload down a separate path, with the reason in the error field, instead of stopping the pipeline.

Logging

The log tells you which server the transform ended up on, and what it did:

Detailed : the connection, the fields in use, every uploaded file with its size and duration, and a summary at the end
Debug : one line per row with the source and the target, whether the remote folder was created, and what happened to the source file afterwards

Metadata Injection

All options of this transform support metadata injection.

Migrating from Kettle/PDI

The Kettle SFTPPut step carries the server, the credentials and the proxy settings in the step itself. The Kettle import moves those settings into an SFTP Connection named after the server, points the transform at it, and reuses one connection for all the steps which talked to the same server.