As always at the start of another new month, let’s have a look at what happened at Hop over the last month. Here’s the recap for January 2021!
Apache Hop (Incubating) 0.60 Released
Apache Hop (Incubating) 0.60 was released in late January.
With 0.60 being the first Hop release as an Incubating ASF project, we focused on adopting the "Apache Way", and made this a source code only release.
Check all details in the release announcement.
Since the release of 0.60, the last license incompatibilities have been resolved: all non-compliant source code has been cleanup up or removed, Hop is now 100% ASF compliant.
Password Encoding
Metadata password encoding has now been made pluggable and is synced with the rest of the Hop platform with the AES (Advanced Encryption Standard) passwords implementation as a first plugin.
Translation improvements
Having Hop available in one’s native language significantly improves developer productivity. That is why we introduced the Hop Translator tool very early after Hop’s introduction, and have now simplified the process of adding and translating i18n keys for developers and translators.
Developers can now specify i18n keys as i18n:package:key, for example in the annotation below:
@GuiMenuElement(
root = ID_MAIN_MENU,
id = ID_MAIN_MENU_FILE,
label = "i18n::HopGui.Menu.File",
parentId = ID_MAIN_MENU)Check the internationalisation documentation for more details.
On that page, you’ll also find information about the Translator’s use. Check the translation contributor guide to check how you can start contributing translations in your native language.
New Plugins
The list of available plugins in Hop keeps growing. Thanks Matt, Nicolas and Sergio for January’s batch of new plugins:
-
Action(s):
-
Passwords (see up):
-
Transforms:
Kettle/PDI Importer
From the start in late 2019, Hop’s goal was to be an independent platform. Hop started from the Kettle/PDI 8.2.0.7 code base, but both platforms have moved in different directions and are not compatible.
Users and developers who want to migrate their Kettle/PDI projects to Hop are now one File → Import from Kettle/PDI click away from doing so. Specify your Kettle/PDI jobs and transformations folder, a Hop project or folder to import to, and you’re done. Optionally, kettle.properties, shared.xml and simple-jndi’s jdbc.properties can be imported to Hop variables and relational database connections as well.
Check the documentation for more details and don’t forget to create a JIRA ticket if you find any issues.
Various
Continued L&F improvements
We’ve continued to tweak Hop Gui’s look and feel, among other things with improved icons resolution on Mac.
Project improvements
We continue to work on project and environment improvements. After lengthy discussions on the chat ~dev channel and on the mailing list, we made a couple of changes. Projects are now optional, have a default and can inherit from other projects.
Community
The Hop community continues to grow:
Check out the complete list of committers and contributors.
Without community contribution, Hop is just a coding club! Please feel free to join, participate in the discussion, test, file bug tickets on the software or documentation, … Contributing is a lot more than writing code.
Check out our contribution guides to find out more.
JIRA Tickets
The full list of issues that had activity over the last month is:
Resolved (71)
| Issue | Summary | Components | Created | Updated |
|---|---|---|---|---|
Renaming database connection creates duplicates | GUI | 2020-10-04 | 2021-01-16 | |
"Clicking ""get more rows"" closes preview dialog" | GUI | 2020-10-04 | 2021-01-16 | |
find an APL 2.0 alternative for the GPL licensed EDT FTP client in the Put FTP action | Actions | 2020-10-04 | 2021-01-27 | |
Fix the beam-demo project | Beam | 2020-10-04 | 2021-01-25 | |
Native core plugins a registered twice | 2020-12-04 | 2021-01-29 | ||
Enable pritty print on JSON output | Transforms | 2020-12-16 | 2021-01-20 | |
Single click mode: moving a transform or action can cause edit | GUI | 2020-12-21 | 2021-01-04 | |
App icon on macOS looks differently | GUI | 2021-01-01 | 2021-01-22 | |
Translator: implement multi-line code scanner | Translations | 2021-01-01 | 2021-01-13 | |
Add Injection support to JSON Output transform | Transforms | 2021-01-02 | 2021-01-27 | |
Deprecate smart-json and migrate to jackson in JSON transforms | Transforms | 2021-01-02 | 2021-01-27 | |
Add documentation to Enhanced JSON Output Transform | Documentation | 2021-01-02 | 2021-01-22 | |
Replace JFace MessageDialogWithToggle | GUI, Hop Web | 2021-01-02 | 2021-01-04 | |
Make deprecated variables (linesRead, linesWritten,linesOutput.. ) private in BaseTransform | 2021-01-02 | 2021-01-12 | ||
Use environment variable HOP_PLUGIN_BASE_FOLDERS to avoid having to pass it as a system property | CLI | 2021-01-02 | 2021-01-09 | |
Get Fields button not working on JSON Output Transform | Transforms | 2021-01-03 | 2021-01-20 | |
Exporting to SVG should overwrite the file if it exists | GUI | 2021-01-03 | 2021-01-03 | |
"change default of abort transform to ""abort with error""" | Transforms | 2021-01-04 | 2021-01-10 | |
Remove old Kettle samples, code and welcome pages | 2021-01-04 | 2021-01-04 | ||
add ASF header to adoc/md files | Documentation | 2021-01-04 | 2021-01-04 | |
New design for worklow/pipeline graph | GUI | 2021-01-04 | 2021-01-05 | |
cleanup RAT excludes | Build | 2021-01-05 | 2021-01-07 | |
Context dialog has problems when using the scrollbar | 2021-01-05 | 2021-01-27 | ||
Clean up old transformation and job files | 2021-01-06 | 2021-01-16 | ||
change selected text color | Website | 2021-01-07 | 2021-02-02 | |
add Disclaimer-wip to repo | 2021-01-08 | 2021-01-10 | ||
Update headers to be apache conform | 2021-01-08 | 2021-01-10 | ||
remove edtftpj dependency | 2021-01-08 | 2021-01-27 | ||
no object DCH for MIME type multipart/mixed error sending email in pipeline | 2021-01-09 | 2021-01-16 | ||
hop-assemblies-debug no longer works | Build | 2021-01-09 | 2021-01-10 | |
Update Apache Beam API to 2.27.0 | Beam | 2021-01-10 | 2021-01-13 | |
Split hop does not work | GUI, Hop Web | 2021-01-11 | 2021-01-19 | |
Error display tooltip when transformation failed | GUI | 2021-01-12 | 2021-01-12 | |
Field splitter: add support for escape string | Transforms | 2021-01-14 | 2021-01-16 | |
"""Open file"" dialog does not show expected files" | GUI | 2021-01-14 | 2021-01-16 | |
Test DB connection does not use variables correctly | GUI | 2021-01-14 | 2021-01-18 | |
Concat fields: doesn’t work with lazily converted data | Transforms | 2021-01-15 | 2021-01-16 | |
Create an action to run pipeline unit tests | Actions | 2021-01-15 | 2021-01-16 | |
Hop icons are not centered when enlarging the icons with 'Icon size in workspace' | GUI | 2021-01-15 | 2021-01-16 | |
"When using ""workflow executor"" the metadata is not loaded correctly" | Transforms | 2021-01-15 | 2021-01-16 | |
Concat fields transform throws an error when running inside a container | Integration Testing, Transforms | 2021-01-16 | 2021-01-18 | |
Pipeline errors downstream with table Input using connection variables | 2021-01-16 | 2021-01-18 | ||
The data icon is not positioned correctly | GUI | 2021-01-16 | 2021-01-17 | |
Error creating relational database connection | Metadata | 2021-01-17 | 2021-01-18 | |
JSON Input: Select Fields button doesn’t do anything | Transforms | 2021-01-18 | 2021-01-19 | |
Create action to execute AS400 command | Actions | 2021-01-18 | 2021-01-19 | |
"Rename component ""Web Hop"" to ""Hop Web"" on Jira" | Hop Web | 2021-01-19 | 2021-01-19 | |
"Add ""Metadata structure of stream"" plugin" | Transforms | 2021-01-19 | 2021-01-29 | |
runtime variable not being picked up | 2021-01-19 | 2021-01-19 | ||
Json input still uses system file browser | Transforms | 2021-01-19 | 2021-01-27 | |
Implement global bookmarks for the file dialogs | GUI | 2021-01-20 | 2021-01-27 | |
Removed StyledText from the Hop code | GUI | 2021-01-20 | 2021-01-22 | |
Workflow and Action code cleanup | 2021-01-20 | 2021-01-28 | ||
Reference to wrong doc file in Enhanced JSON Output | 2021-01-21 | 2021-01-27 | ||
Enhanced JSON Output still uses system file dialog | Transforms | 2021-01-21 | 2021-01-27 | |
Add tool to clear database cache | GUI | 2021-01-22 | 2021-01-26 | |
PatternSyntaxException when importing kettle jobs and transformations on Windows | Import | 2021-01-24 | 2021-02-02 | |
Remove annotation property 'i18nPackage' | 2021-01-25 | 2021-02-01 | ||
JSON Output still uses file system dialog | 2021-01-27 | 2021-01-27 | ||
XML Output still uses file system browser | 2021-01-27 | 2021-01-27 | ||
XML Input stream still uses file system browser | 2021-01-27 | 2021-01-27 | ||
Get Data from XML still uses file system browser | 2021-01-27 | 2021-01-27 | ||
Remove Sleak and ImageUtil from source code | GUI | 2021-01-27 | 2021-01-27 | |
Create a plugin to encrypt passwords using AES | API, Metadata | 2021-01-27 | 2021-01-29 | |
Remove unused class CertificateGenEncryptUtil | API | 2021-01-28 | 2021-01-29 | |
Some plugins/ folders are scanned twice | API | 2021-01-28 | 2021-01-29 | |
Hop plugin loading is timed but this is never used | API | 2021-01-28 | 2021-01-29 | |
Remove IPluginFolder | 2021-01-30 | 2021-02-01 | ||
fix typos in website | Website | 2021-01-31 | 2021-02-02 | |
Translator does not work under Windows | 2021-01-31 | 2021-02-01 |
In Progress (7)
| Issue | Summary | Components | Created | Updated |
|---|---|---|---|---|
In Pipeline and Workflow screens the select action dialog box cannot be moved around the screen or resized. | GUI | 2020-10-04 | 2021-01-16 | |
"Hop needs a default ""Project"" and ""Local Run Configuration""" | 2020-10-04 | 2021-02-02 | ||
Improve first-time usage experience | GUI | 2020-11-12 | 2021-02-01 | |
See what files are still needed in the client zip to be release compliant | 2021-01-26 | 2021-01-26 | ||
Ensure that Hop complies with ASF encryption policy | 2021-01-27 | 2021-01-28 | ||
Add UDJE (User Defined Java Expression) to ETL Metadata Injection step | 2021-01-28 | 2021-02-01 | ||
Options Dialog : ConfigPlugins are no longer showing up | GUI | 2021-01-30 | 2021-02-01 |
Open (42)
| Issue | Summary | Components | Created | Updated |
|---|---|---|---|---|
Integrate WebSpoon code | Hop Web | 2020-10-04 | 2021-01-16 | |
Move JDBC properties in BaseDatabaseMeta.class to the database plugins | Database | 2020-10-04 | 2021-01-16 | |
Wrong definition of default Locale in HopGui | GUI | 2020-10-04 | 2021-01-16 | |
Create audit manager | 2020-10-04 | 2021-01-16 | ||
"Replace ""Select Values"" by 3 new Transforms" | Transforms | 2020-10-04 | 2021-01-16 | |
Epic to hold all remaining transforms that have to be ported | Transforms | 2020-10-04 | 2021-01-16 | |
In the Pipeline Run Configuration show which fields are mandatory | GUI | 2020-10-04 | 2021-01-16 | |
Environments with tilde in filename are created in unexpected place on *nix systems | GUI | 2020-10-04 | 2021-01-16 | |
Change of database connection not applied | 2020-10-04 | 2021-01-16 | ||
Data set: Show error if filename or path is wrong | 2020-10-04 | 2021-01-16 | ||
When setting HOP_CONFIG_DIRECTORY, currently no config.json is created | 2020-10-04 | 2021-01-16 | ||
Beam: Error converting Hop data to string lines | Beam | 2020-10-04 | 2021-01-16 | |
"Exception in thread ""Thread-52"" java.lang.RuntimeException: Error starting the Beam pipeline" | Beam | 2020-10-04 | 2021-01-16 | |
Adding run config without name causes exception | GUI | 2020-10-04 | 2021-01-16 | |
Rename package org.apache.hop.pipeline.transform.errorhandling | 2020-10-04 | 2021-01-16 | ||
Keyboard shortcut for Edit Metastore Element | GUI | 2020-10-04 | 2021-01-16 | |
Migrate Generic Connection to Plugin | Database | 2020-10-04 | 2021-01-16 | |
More consistency to hop-conf.sh options/flags | CLI | 2020-10-04 | 2021-01-16 | |
Document the new VFS plugins | 2020-10-04 | 2021-01-16 | ||
Text unreadable in Options dialog | GUI | 2020-10-04 | 2021-01-16 | |
Variable from environment config only work after restart | GUI | 2020-10-04 | 2021-01-16 | |
Fix button positions in dialogs | 2020-10-04 | 2021-01-16 | ||
Remove options from menu when not available for transform | GUI | 2020-10-04 | 2021-01-16 | |
when selecting a folder and not a file an error is thrown | 2020-10-04 | 2021-01-16 | ||
As a data engineer I would like to perform data validation | Transforms | 2020-10-04 | 2021-01-27 | |
Postgres bulk loader leaks connections | Transforms | 2020-10-04 | 2021-01-16 | |
implement metadata injection for all transforms that used the deprecated step api | Transforms | 2020-10-04 | 2021-01-16 | |
Bug seen during find | 2020-10-04 | 2021-01-16 | ||
port sample rows step to transform | Transforms | 2020-10-04 | 2021-01-16 | |
Replace javax/mail/Address dependency | Transforms | 2020-11-05 | 2021-01-16 | |
When reloading project variables they are not refreshed | Metadata | 2021-01-17 | 2021-01-23 | |
Projects: open objects are not remembered in the metadata perspective | Metadata, Projects | 2021-01-17 | 2021-01-20 | |
HopWeb: server crash during preview | Hop Web | 2021-01-22 | 2021-01-22 | |
Hop Web: not possible to see data of a transform | Hop Web | 2021-01-22 | 2021-01-22 | |
attach POM to the correct phase | 2021-01-25 | 2021-01-25 | ||
Provide some samples for Enhanced JSON Output | 2021-01-27 | 2021-01-27 | ||
NPE in file open dialog | GUI | 2021-01-27 | 2021-01-27 | |
Option to build minimal Hop package | 2021-01-29 | 2021-01-29 | ||
Support for Objects and Array data types | 2021-01-29 | 2021-01-29 | ||
"Picking ""Create hop"" eventually brings up action picker again" | GUI | 2021-02-02 | 2021-02-02 | |
Unit tests UI issues | 2021-02-02 | 2021-02-02 | ||
Port MonetDB Bulk Loader to Hop | 2021-02-02 | 2021-02-02 |
Closed (6)
| Issue | Summary | Components | Created | Updated |
|---|---|---|---|---|
JSON Output: ERROR: Output fieldname that will contain value is empty! | Transforms | 2020-10-04 | 2021-01-16 | |
port formula step to transform plugin | Transforms | 2020-10-04 | 2021-02-02 | |
Add RAP fragment to the master branch | 2021-01-02 | 2021-01-03 | ||
Remove JFace from hop-assemblies-libs and add it to hop-client | GUI, Hop Web | 2021-01-03 | 2021-01-16 | |
Remove Servlet API jars | Hop Server | 2021-01-04 | 2021-01-25 | |
Detect actions / transforms deprecated by java annotation @Deprecated | 2021-01-27 | 2021-01-28 |