Apache Hop

Security

When thinking about security it’s always worth checking on any critical vulnerabilities the Hop codebase itself might suffer from. Go to https://hop.apache.org/security/ to find out more.

Then obviously there are things our users can do to limit security risks in general listed below.

The goal of Hop as a project is to be useful for you, the data engineer. As a project we provide you with many powerful tools that can do as much as you request it. In that regard it’s very much like a programming language or scripting tool in the way that you can create, modify and delete things at your hearts content. Please understand that this is a feature, not a security risk.

Securing your work

When dealing with sensitive information that might be needed to interact with databases, remote folders, websites, and so on, make sure to properly protect the secrets being used. Here are things you can do:

Not in your metadata!

Please, never store your secrets in your metadata. Always use variables to reference to them indirectly. This allows flexibility as to the level of security you require. For a local database on your own laptop it’s fine to simply obfuscate the password or even leave it in cleartext. That’s something you can define in a separate environment configuration file. For a shared development, test or production system, you need the variable to point to an obfuscated or encrypted value, or indeed one stored in a key vault.

Obfuscate

This is the absolute minimum you need to do. Make sure to at least obfuscate your secrets with a standard hop encrypt command. Please understand that it’s hard but not impossible to get the original value from the generated obfuscation code. For that reason it’s important to store the obfuscated secrets in a safe place on your systems.

Encrypt

Consider setting up asynchronous encryption to hide your secrets with the AES two way password encoder. Please understand that in this scenario you need to protect the value stored in the HOP_AES_ENCODER_KEY variable. Again, keep it in a safe place like a k8s key vault or a well protected file on your system.

Use a Key Vault

Finally, you can store your secrets in a so-called key vault. For a list of the supported key-vaults and on the usage of variable resolvers to get direct access to the secrets, see: Variable Resolvers. Again, just using a key vault is insufficient: make sure you protect the secrets (keys, users, passwords) to connect to the key vault you’re using.

AI Assistant: secrets, personal information, and minimum send

The AI Assistant sends the open pipeline or workflow context to a language-model service. Anything hard-coded in that file (or in its logs and check results) can leave your machine.

Set up the project before you enable AI:

Never hard-code secrets or personal information in pipelines, workflows, metadata, notes, SQL, or sample rows.
Put values in environment configuration files as variables. Keep those files outside the project repository.
Prefer a keystore or vault, referenced with a variable resolver.
Name secret variables so they contain PASSWORD, SECRET, TOKEN or PASSWD. Hop Gui masks those in the Ctrl-Space picker, and the assistant redacts matching XML/JSON keys.
Keep personal records (names, emails, national ids, row samples) out of notes, Data Grids and log messages.

On each question, use the session context inclusion checkboxes to send only what is needed: check results, plugin catalog, execution log, full XML. That cuts token cost and the chance of sharing sensitive data. See Context inclusions.

Secure your Hop Server

If you’re running the Hop Server on a Kubernetes or VM constellation you can secure network access from and to the clients that way. Otherwise, consider setting up SSL communication to prevent anyone from listening in on the network traffic. Please see the SSL Configuration documentation here.

Remove unused plugins

Every organisation has its own typical mix of deployed technologies. As a best practice, remove any plugin that supports technology that you’re not using. Go through the list of plugins in the plugins/ folder and remove anything that you’re not going to need. Focus especially on older technology that might contain drivers that perhaps haven’t been updated in a while. You can safely remove any folder in the plugins/ folder.

Scripting

The scripting actions and transforms in Hop are capable of doing just about anything if you know how. It’s not a best practice to do "anything", in fact the best practice is to make use of the transforms as little as possible to make the maintenance load of your data integration solutions as light as possible. If you can’t avoid using scripting, put some process in place to code review the scripts and keep them simple and transparent.

Remove unused scripting plugins

The JavaScript, Script, and other coding transforms are powerful but can execute arbitrary code, potentially leading to code injection or data exfiltration risks.

Scripting transforms can execute arbitrary code. If you’re reading data from untrusted sources or allowing user input to influence script logic, you risk code injection attacks, data exfiltration, or unintended side effects on your system.

Removing the plugins

If you don’t want to use scripting transforms, you can remove the associated plugins:

shell

# Navigate to your Hop installation directory
cd $HOP_HOME/plugins

# Remove the scripting-related plugin directories
rm -rf transforms/script/ # Groovy, Python
rm -rf transforms/javascript/ # JavaScript
rm -rf transforms/janino/ #UDJC, UDJE, Java Filter

# Restart Hop Gui or Hop Server to apply changes

Before removing plugins, verify that no workflows or pipelines depend on the removed transforms. Broken workflows or pipelines will fail to execute.

Security best practices for scripting

If you do use scripting transforms, follow these guidelines:

Limit script scope - Keep scripts focused and minimal. Prefer native Hop transforms when possible.
Code review process - Establish a formal review process for all scripts used in production workflows.
Audit script execution - Log when and how scripts are executed, especially in production environments.
Use sandboxed environments - Where possible, run scripts in isolated environments with limited system access.