After more than half a year of hard work, we felt the time was right to show the world what we have been working on.

We’re happy to announce to availability of our 0.10 preview release. Please take it for a spin. If you find a bug, please create a bug ticket, if there’s functionality you’d like to see, please create a feature request.

Direct download here

Getting Started Guide

Release Notes

What is Project Hop

Project Hop’s goal is to provide an easy, flexible and metadata-driven way of data processing and data orchestration. We want you to spend as little time as possible on technical details, so you can focus on processing your data and orchestrating your pipelines.

We used the Kettle (Pentaho Data Integration) 8.2.0.7 code base to start building our architecture. Kettle was the start of a journey. We’ve come a long way from that starting point: a lot of the code base has been removed or changed, and a lot of new functionality has been added.

Let’s have a look at what to expect in this release.

Code Cleanup, Refactoring

We started with a huge spring cleaning session: Karaf, XUL, repositories and the old user interface (Spoon) are all gone.

Dependencies have been updated to the most recent versions available.

A lot of refactoring was required to bring the Hop code base up to date, more flexible and more pluggable. We’ve also massively refactored the code to reflect the Hop concepts.

Hop Concepts

Hop’s tools reflect its Architecture, check out the Concepts page.

Our main concepts are:

  • Workflows manage the orchestration of your Hop code. Workflows have actions as their main items of work.

  • Pipelines are where the real work is done. Pipelines consist of a number of transforms that each perform atomic tasks (read, transform, write) in your pipeline.

The tools we have available at the moment are:

  • hop-gui: a visual IDE to design, test and run your workflows and pipelines.

  • hop-run: a command line tool to run your workflows and pipelines

  • hop-server: a lightweight web server to run your workflows and pipelines over a REST interface

  • hop-translator: a helper tool to translate Hop into your language

Plugins

We want to make Hop as robust and flexible as possible. We’ve done a lot of work to move Hop to a kernel-based architecture. To get there, we’ve moved all non-essential functionality out of the Hop engine and into plugins. Going forward, every single piece of functionality will be supported by the engine, but implemented as a plugin.

The current status of functionality that has been ported from the engine to plugins is:

  • Database plugins: all done (39)

  • Workflow actions: all done (53)

  • Transform actions: 60 done (about 80 to go, NOTE: porting the transforms to plugins is more complex since some of them need significant code refactoring.

UI

Hop comes with a completely new UI. Simplified, flexible and future proof.

Another one of our goals is to provide 100% UI coverage for all Hop functionality. Changing the behavior of your workflows and pipelines should be possible, but never required.

The Workflow editor:

Interface 1

The Transform chooser dialog:

Interface 1

Partitioning:

Partitioning

Extra options on transform action:

Transform action

Translations

Hop is available in the following languages:

  • German (de_DE)

  • American English (en_US)

  • Argentinian Spanish (es_AR)

  • French (fr_FR)

  • Italian (it_IT)

  • Japanese (ja_JP)

  • Korean (ko_KR)

  • Dutch (nl_NL)

  • Brazilian Portugese (pt_BR, kindly contributed as the first new language in Hop!)

  • Chinese (zh_CN)

Check the Translation Contribution Guide to add or improve the Hop translation for your own native language.

Translator

We also included the Hop Translator for community members who’d like to translate Hop to their native language, or complete the current translations for the project.

Translator

Check the Translation Contribution Guide to add or improve the Hop translation for your own native language.

Documentation

Documentation is a first class citizen at Project Hop. All documentation is written in asciidoc and is treated just like any other piece of code, so feel free to create bug tickets or feature request for documentation, just like you would for any other part of the project.

Future

Work isn’t done with this preview release.

Next up are pluggable runtime engine. The first engine we’ll support will be Apache Beam, but there will be many more.

We intend to start the Apache Incubation process soon. We strongly believe the move to Apache Hop will increase Project Hop’s adoption. We are looking forward to working with the Apache Software Foundation and to integrating with the great software they provide.

Call For Contributors

Project Hop is a team effort, we need your help to make this a success!

Contributing is much more than writing code. A couple of ways you can help out are

Check out the Contribution Guide to find out how you can contribute.

Contributions in any shape or form are greatly appreciated!