The Apache Hop team just released version 1.0.0 and became a top level project at the Apache Software Foundation. The immediate next goal for us is to release version 1.1.0 with almost 200 fixes and improvements.

Here are a few directions that we want to explore for the future of our software:

  • Set up a software marketplace where 3rd party plugins can be found

  • Pluggable field expressions in transforms and actions

  • Integration with Apache Airflow both in Airflow and as a new Workflow engine plugin

  • Modularisation of the Apache Beam plugins

  • Further unification of Beam plugins, for example make the Beam BigQuery Input plugin work in non-Beam engines as well

  • Finish conversion of all plugins to generic XML serialization to allow for JSON/YAML/…​ metadata serialization of pipelines and workflows

  • Create a new GUI for executing, previewing and debugging pipelines and workflows with care for long-running and streaming workloads

  • Create a monitoring and logging network service capable of accepting operational metadata from local and remote Hop executions, even on clusters (Beam)

  • Convert the Neo4j logging perspective to a generic logging perspective with room for plugins exposing the underlying strength of the logging sink.