With 2020 coming to a close, we’ve started preparing for 0.50, which will be our first Apache release. We’ll have more news on that soon!

At the start of the final month of 2020, let’s have a closer look at what happened at Apache Hop in November.

Apache Hop Incubation - infrastructure

Our migration to the Apache Software Foundation infrastructure is now done. All references to Project Hop have been changed to Apache Hop (Incubating).

All software and documentation code have been moved to apache-hop-* repositories. All builds have been ported or recreated, some additional features have been added (e.g. PR builds over GitHub Actions). Thank you Hans to get all of this to work seamlessly!

New Plugins

Quite a number of new plugins are now available in Hop:

Repeat End-Repeat

  • Transforms

    • MongoDB Input and Output

    • CPython: Mark Hall ported the CPython plugin he wrote for Kettle/Pentaho to Hop.

    • Salesforce:

      • Salesforce Delete

      • Salesforce Input

      • Salesforce Insert

      • Salesforce Update

      • Salesforce Upsert

Repeat End-Repeat

Documentation

As mentioned a couple of times in previous updates, documentation will be work in progress for the foreseeable future. However, there is progress!

We’ve received some additions and updates to the user manual.

The majority of the work was done on the developer documentation (thanks, Hans, Hiromu, Mahendran, Matt!):

As always, ping us on mattermost to discuss documentation, and create tickets if you find any missing or incorrect information in the docs. Documentation is code, we take bugs in the documentation just as serious as bugs in the code.

Community

The Hop community continues to grow:

  • chat: 108 registered members (up from 104) join

  • LinkedIn: 309 followers (up from 263) follow

  • Twitter: 233 followers (up from 219) follow

  • YouTube: 40 subscribers (up from 38) subscribe

  • Facebook: 20 followers (up from 19) like

The following people have been added as committers:

  • Peter Fabricius

  • Sergio Ramazzina

Check out the complete list of committers and contributors.

Without community contribution, Hop is just a coding club! Please feel free to join, participate in the discussion, test, file bug tickets on the software or documentation, …​ Contributing is a lot more than writing code.

Check out our contribution guides to find out more.

Various

The full list of issues that had activity over the last month is:

Resolved (76)

Issue Summary Components Created Updated

HOP-1733

Update MSSQLNative documentation with info on how to use integrated security

Database, Documentation

2020-10-04

2020-11-15

HOP-1757

Error thrown when hop.properties doesn’t exist

2020-10-04

2020-11-23

HOP-1847

Editing Pipeline Run Configuration causes error

2020-10-04

2020-11-23

HOP-1896

Refactor UuidUtil

2020-10-04

2020-11-26

HOP-1905

MemoryGroupByMeta not working in Beam

Beam

2020-10-04

2020-11-23

HOP-1943

hop-conf.sh: Create config.json if missing

2020-10-04

2020-11-23

HOP-1997

Transform context option to specify row distribution doesn’t work

Transforms

2020-10-04

2020-11-23

HOP-2013

Project not correctly created

GUI

2020-10-04

2020-11-23

HOP-2045

Update Generic Database connection Documentation

Database, Documentation

2020-10-04

2020-11-15

HOP-2057

update environments page

Documentation

2020-10-04

2020-11-15

HOP-2080

Remove or adapte file resources/org/apache/hop/ui/core/dialog/license/license.txt

GUI

2020-10-04

2020-11-25

HOP-2082

port Stream Schema Merge plugin to Hop transform

Transforms

2020-10-04

2020-11-18

HOP-2090

Stream Schema Merge - NPE

Transforms

2020-10-04

2020-11-18

HOP-2101

In a fresh copy of hop, a pipeline will fail to run without a project.

2020-10-04

2020-11-23

HOP-2102

If you close the Preview Rows dialog using the "X", the preview will keep running with no way to cancel (ever).

2020-10-04

2020-11-18

HOP-2103

Without a project; Text Input Step fails to record the whole file name for browsing and selecting files.

2020-10-04

2020-11-25

HOP-2121

Update base doc url to https://hop.apache.org/

Documentation, GUI

2020-10-18

2020-11-15

HOP-2123

Code cleanup: rename variables with bad style

2020-10-19

2020-11-09

HOP-2126

Parameter HOP_MAX_LOG_SIZE_IN_LINES is set too low

2020-10-22

2020-11-12

HOP-2127

Workflows do not correctly capture their log channel

2020-10-22

2020-11-09

HOP-2128

Context dialog for Notes is missing in workflows

2020-10-23

2020-11-25

HOP-2134

Add support for ordered categories for context actions, expose in context dialog

GUI

2020-10-27

2020-11-16

HOP-2135

Null pointer exception when running Shell action in a workflow

Actions

2020-10-27

2020-11-12

HOP-2136

Options: Font size changes are not honored

GUI

2020-10-30

2020-11-23

HOP-2137

Zoom factor is incorrectly calculated on Windows

GUI

2020-10-30

2020-11-12

HOP-2140

Exception raised when running Hop Translator

2020-11-03

2020-11-12

HOP-2146

AuditManagerTest fails

2020-11-06

2020-11-09

HOP-2152

Enum HopExtensionPoint use Job name

2020-11-10

2020-11-30

HOP-2156

AbstractMetaTest.testMultithreadHammeringOfListener causes OOM

2020-11-11

2020-11-12

HOP-2160

Wrong name of project and enviroment after update

GUI

2020-11-12

2020-11-25

HOP-2164

Add format selector in Data Set creation

GUI

2020-11-12

2020-11-12

HOP-2165

Add format selector in File Definition creation

GUI

2020-11-12

2020-11-12

HOP-2167

ContextDialog NPE

GUI

2020-11-12

2020-11-17

HOP-2168

Store run configuration names per file

GUI

2020-11-12

2020-11-17

HOP-2169

Random Value transform generates data of wrong class

2020-11-12

2020-11-16

HOP-2170

Evaluation Action: parent_workflow unavailable

Actions

2020-11-13

2020-11-23

HOP-2173

Make it easy for plugins to get lists of names from other plugins

GUI

2020-11-13

2020-11-17

HOP-2175

Translation issues

GUI

2020-11-16

2020-11-23

HOP-2180

Failed to show filename, file content & fields

GUI

2020-11-16

2020-11-23

HOP-2183

Error pasting from clipboard

GUI

2020-11-16

2020-11-24

HOP-2184

Canvas grid option doesn’t update after saving

GUI

2020-11-16

2020-11-24

HOP-2185

Font and color options do not update after setting

GUI

2020-11-16

2020-11-24

HOP-2187

ContextDialog to create an item: set Workflow and pipeline in a Basic category

2020-11-16

2020-11-18

HOP-2188

Add RAT to PR Build and make it pass

Build

2020-11-17

2020-11-17

HOP-2189

Port the Salesforce transforms

Transforms

2020-11-17

2020-11-17

HOP-2192

Undoing delete of a note changes font color

2020-11-17

2020-11-17

HOP-2193

NPE in Copy Rows To Result

Transforms

2020-11-17

2020-11-18

HOP-2194

ITransform declaring meta or data causing NPEs

Transforms

2020-11-17

2020-11-18

HOP-2195

rename hop-assemblies-client to hop-client

Build

2020-11-17

2020-11-18

HOP-2196

Split the File/New actions into File and Metadata categories

GUI

2020-11-17

2020-11-18

HOP-2197

Changing variable in project dialog requires restart

GUI

2020-11-17

2020-11-19

HOP-2198

Workflow Transform not working

2020-11-17

2020-11-18

HOP-2200

Remove the Hop Lifecycle listener plugin type

API

2020-11-18

2020-11-26

HOP-2201

Workflow executer Transform does not show logging

Transforms

2020-11-18

2020-11-18

HOP-2202

Passing parameters using the "Copy results to parametrs" option does not work

Actions, Transforms

2020-11-18

2020-11-19

HOP-2208

Make ContextDialog more friendly for plugins

GUI

2020-11-19

2020-11-19

HOP-2209

ContextDialog: remove excess spacing and add "clear search" icon

GUI

2020-11-19

2020-11-19

HOP-2210

Add m2 caching to PR build

Build, Infrastructure

2020-11-19

2020-11-22

HOP-2211

Port the MongoDB plugins

Transforms

2020-11-19

2020-11-23

HOP-2214

Issue with variable resolution of empty values

Transforms

2020-11-20

2020-11-23

HOP-2215

Creating project with relative path in home folder creates NPE

CLI

2020-11-20

2020-11-24

HOP-2216

hop-run, return exit code 1 when a workflow aborts with error

CLI

2020-11-22

2020-11-28

HOP-2219

create mongoDB documentation

Documentation

2020-11-23

2020-11-24

HOP-2221

Clear custom options in the options dialog doesn’t save to file

GUI

2020-11-24

2020-11-24

HOP-2222

Creating new project fails

GUI

2020-11-24

2020-11-24

HOP-2224

Renaming metadata entries creates a copy

Metadata

2020-11-24

2020-11-25

HOP-2227

The preview rows dialog should not pop up when dragging

GUI

2020-11-25

2020-11-27

HOP-2230

The Metadata Injection dialog references methods of specification

Transforms

2020-11-25

2020-11-27

HOP-2232

Port the repeat-until actions

2020-11-26

2020-11-27

HOP-2233

Project variables are not passed to a new workflow

Workflows

2020-11-26

2020-11-27

HOP-2234

Values are not passed to parameters in the Workflow Executor transform

Transforms, Workflows

2020-11-26

2020-11-27

HOP-2236

Action dialogs are not searched for in the plugin classpath

API

2020-11-27

2020-11-27

HOP-2237

Resizing the file dialog collapses columns

GUI

2020-11-27

2020-11-27

HOP-2238

Add search to metadata injection Transform

Transforms

2020-11-27

2020-11-27

HOP-2240

The JavaScript action still uses entry, doc says action

Actions

2020-11-27

2020-11-27

HOP-2241

A stopped workflow will think it’s still running

Workflows

2020-11-27

2020-11-27

In Progress (9)

Issue Summary Components Created Updated

HOP-2132

Action with Connection line not correctly translated

Actions, Translations

2020-10-25

2020-11-30

HOP-2191

Check and update notice,license,disclaimer,…​

2020-11-17

2020-11-24

HOP-2203

Create About page

GUI

2020-11-18

2020-11-18

HOP-2212

Create first version of Integration Testing

Build, Infrastructure, Integration Testing

2020-11-20

2020-11-20

HOP-2217

Port the Cassandra plugin

Transforms

2020-11-22

2020-11-29

HOP-2226

Document how to set up a developer environment

Documentation

2020-11-25

2020-11-25

HOP-2235

Create a perspective to manage metadata

GUI

2020-11-26

2020-11-29

HOP-2239

When executing a workflow manually set variables are not applied

GUI

2020-11-27

2020-11-27

HOP-2244

Reverse sort the projects drop-down list by use date

GUI

2020-11-30

2020-11-30

Open (42)

Issue Summary Components Created Updated

HOP-1593

Review/Refactor getValueFromSQLType

Database

2020-10-04

2020-11-09

HOP-1605

API should use boolean "isXXXX()" getter naming convention

Database

2020-10-04

2020-11-09

HOP-1835

Create Help- About menu entry

GUI

2020-10-04

2020-11-18

HOP-2054

Fix the beam-demo project

Beam

2020-10-04

2020-11-12

HOP-2071

Create Fake Data Transform Documentation

Documentation

2020-10-04

2020-11-03

HOP-2104

Hop needs a default "Project" and "Local Run Configuration"

2020-10-04

2020-11-12

HOP-2141

NullPointerException raised on hop-server workflow start

Hop Server

2020-11-05

2020-11-05

HOP-2142

Replace javax/mail/Address dependency

Transforms

2020-11-05

2020-11-23

HOP-2143

Not found from Search option

Documentation

2020-11-06

2020-11-23

HOP-2144

Git Repository option is missing

Documentation

2020-11-06

2020-11-06

HOP-2145

"Synchronise name with" option doesn’t work (Workflow properties)

GUI

2020-11-06

2020-11-29

HOP-2147

hop-server.bat has wrong line ending char

Hop Server

2020-11-09

2020-11-09

HOP-2148

NullPointerException raised on stopping an inactive workflow

Hop Server

2020-11-10

2020-11-10

HOP-2149

Standardize option name "Synchronise name with filename"

GUI

2020-11-10

2020-11-17

HOP-2150

Issue editing pipelines and workflows

GUI

2020-11-10

2020-11-16

HOP-2151

Hop-server Object ID not returned in the result from /registerWorkflow request

Hop Server

2020-11-10

2020-11-10

HOP-2154

Error editing Data set with no file name

GUI

2020-11-11

2020-11-26

HOP-2155

Error selecting Edit Hop metadata object

GUI

2020-11-11

2020-11-26

HOP-2159

Flink pipeline engine documentation broken table

Website

2020-11-12

2020-11-12

HOP-2161

MetaStore created with no data

GUI

2020-11-12

2020-11-26

HOP-2162

Error in Ok, Edit and View options when the Data Set has no name

GUI

2020-11-12

2020-11-12

HOP-2163

Edit Repository instead of Git Repository

GUI

2020-11-12

2020-11-12

HOP-2166

Improve first-time usage experience

GUI

2020-11-12

2020-11-12

HOP-2171

Wrong file name

GUI

2020-11-13

2020-11-17

HOP-2172

Disable options as appropriate

GUI

2020-11-13

2020-11-17

HOP-2174

NullPointerException selecting or adding Project or Environment from Search view

GUI

2020-11-16

2020-11-23

HOP-2176

The system removes selection after canceling the creation of a new configuration

GUI

2020-11-16

2020-11-16

HOP-2178

Error selecting file (Text file input)

GUI

2020-11-16

2020-11-23

HOP-2179

Error creating new project

GUI

2020-11-16

2020-11-23

HOP-2181

Error opening file in CSV file input

GUI

2020-11-16

2020-11-23

HOP-2182

Remove dependencies on conflicting licenses

Build

2020-11-16

2020-11-17

HOP-2186

The system doesn’t report existing file name

GUI

2020-11-16

2020-11-26

HOP-2190

IllegalArgumentException Open option (Serialize to file)

GUI

2020-11-17

2020-11-17

HOP-2204

create a menu option to export an entire project as a zip file

GUI

2020-11-18

2020-11-18

HOP-2207

Notify 'Stap Name' is mandatory

GUI

2020-11-19

2020-11-19

HOP-2213

import Kettle/PDI jobs and transformations into Hop

GUI

2020-11-20

2020-11-20

HOP-2218

Copy rows to result should be renamed

Transforms

2020-11-22

2020-11-22

HOP-2223

Doc Build: Images are not copied over from source code

Build, Documentation

2020-11-24

2020-11-24

HOP-2225

Show validation message in modal when no file is selected

GUI

2020-11-24

2020-11-24

HOP-2228

Support Hop on Java 11

API

2020-11-25

2020-11-25

HOP-2229

Configure Java 11 encapsulation

API

2020-11-25

2020-11-25

HOP-2231

Update powermock test libraries to 2.x

API

2020-11-25

2020-11-25

Closed (15)

Issue Summary Components Created Updated

HOP-1740

Zoom dropdown looks missaligned

GUI

2020-10-04

2020-11-07

HOP-1897

All tickets related with Java 11 development

2020-10-04

2020-11-26

HOP-2033

Add latest tag to release docker builds for dockerhub

Containers

2020-10-04

2020-11-18

HOP-2114

Cannot delete selected note with menu Edit > Delete in workflow or pipeline

GUI

2020-10-12

2020-11-17

HOP-2138

Fix sonar issues org.apache.hop.core.xml

2020-11-02

2020-11-07

HOP-2139

IPerspective should return Control not Composite

GUI

2020-11-03

2020-11-09

HOP-2153

Change case for keyboard shortcut on Windows

GUI

2020-11-10

2020-11-12

HOP-2157

MetadataPluginType doesn’t extract image from annotation

GUI

2020-11-11

2020-11-12

HOP-2158

Fix Workflow Hop File Type ID and name

2020-11-11

2020-11-12

HOP-2177

Wrong icon

GUI

2020-11-16

2020-11-30

HOP-2199

Pipeline Executor action error when running for the first time

Actions

2020-11-17

2020-11-18

HOP-2205

TabFolderReorder lost font when TabItem is moved

GUI

2020-11-18

2020-11-19

HOP-2206

Add IHopFileType.CAPABILITY_SAVE_AS

2020-11-18

2020-11-19

HOP-2242

Extension point to detect the change of project

GUI

2020-11-27

2020-11-30

HOP-2243

GuiResource is buggy with same image in two different sizes

GUI

2020-11-29

2020-11-30