Logging and debugging

Logging

Everything that logs owns an ILogChannel, and channels form a parent/child tree that mirrors the execution: pipeline, transform, the database connection inside it. That parentage is what lets Hop show the log of one transform separately from the pipeline it ran in.

Seven levels, from LogLevel:

NOTHING, ERROR, MINIMAL, BASIC, DETAILED, DEBUG, ROWLEVEL

The level is set per run, and a channel logs only what its level allows. ROWLEVEL logs per row and is a debugging tool, not something to leave on.

Attaching a debugger

The launcher scripts in the distribution read HOP_OPTIONS, and each ships with a commented-out JDWP line:

HOP_OPTIONS="${HOP_OPTIONS} -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5009"

Uncomment it, or export HOP_OPTIONS yourself, and attach a remote debugger to that port. This works for hop-gui, hop-run, hop-server and the rest of the scripts.

For running Hop straight from the IDE instead, see Setting up your development environment.

This page is a scaffold. Still to write:

  • Choosing a log level for a message, and the cost of logging on a per-row path — see Performance on the row hot path.

  • How log lines reach the GUI, the execution information location and the server, and why a message can appear in one and not another.

  • Debugging inside a container, and debugging a pipeline running on a remote Hop Server.

  • Getting a thread dump out of a stuck pipeline and reading it: which threads are transforms, which are row sets.

  • What to capture when reporting a bug so it can be reproduced.