Apache Hop
Download
User manual 2.20.0 (pre-release)

User manual

Development Documentation

Edit this page

AI

Apache Hop can advise on pipeline and workflow design through a pluggable AI layer. Completions go through the Language Model Chat transform (hop-transform-languagemodelchat) and langchain4j. No language-model SDK is on the Hop core classpath.

The technology plugin is hop-tech-ai (plugins/tech/ai). AI is disabled until you turn it on.

What ships

  • AI Provider metadata — named model + credentials

  • Configuration perspective → Plugins → AI Assistant — master enable, default provider, extra context notes, context files (default ${PROJECT_HOME}/AGENTS.md), optional full XML

  • AI Assistant workbench — perspective, floating window, and bottom dock

  • Pipeline AI Help and Workflow AI Help — toolbar and context-menu entry points, including review-and-apply hop_proposals graph edits, clipboard XML/JSON, and metadata saves

  • Language Model Chat can optionally use a named AI Provider for connection fields; inline keys remain as fallback

Privacy

AI is off until you turn it on. A prompt that leaves your machine is billed and may be retained by the provider: send the minimum needed to answer the question.

Keep the project in Git (or similar) and create a feature branch before applying AI-driven changes. Graph Undo is not a substitute: metadata saves and pasted transforms or actions cannot be reverted that way. See Version control on the AI Assistant page.

  • Do not hard-code secrets or personal information in pipelines, workflows, metadata, notes, SQL or sample rows. Put them in environment configuration files as variables, or better in a keystore read with a variable resolver. See Secrets and personal information and Best practices: security.

  • API keys stay on AI Provider metadata and are never copied into advisor prompts. Still prefer ${AI_API_KEY} or a resolver expression instead of a pasted key.

  • Prompts redact XML/JSON fields named password, token, api-key and similar. That does not catch literals in SQL, notes or logs.

  • Session context inclusion checkboxes control extra payload (check results, plugin catalog, execution log, full XML, selected metadata). Use only what the question needs: Context inclusions.

  • Extra context notes and listed context files (for example a project AGENTS.md) are sent on every turn when present. Keep secrets out of those files.

  • Advisor plugins may also ship ai-context.md next to the plugin JAR (Hop’s AI plugin does). Those notes are always sent for that advisor.

  • Full pipeline or workflow XML is not sent unless both the session checkbox and Configuration → Allow sending full XML are on.

  • Delete plugins/tech/ai to remove the GUI, metadata type and advisors.

  • Hide individual widgets with disabledGuiElements.xml.

Extending

Two plugin types are registered from HopEnvironment:

  • @AiProviderPlugin / IAiProvider — another backend (including a future OAuth Copilot plugin)

  • @AiAdvisorPlugin / IAiAdvisor — another advisor. Location ids are free-form strings (pipeline graph, Data Vault graph, lineage view, a metadata editor, …). hopper-edw adds Data Vault, Business Vault, dimensional and lineage advisors this way, with their own toolbar and context-menu plugins.

Keep langchain4j and SWT off the IAiAdvisor / IAiProvider interfaces; those live in hop-core. Third-party advisor plugins omit classLoaderGroup and compile against hop-core + hop-ui only. Plugin authors: see the AI advisor plugins page in the development manual.

Keys that older plugins stored under Configuration → AI Assistant (hopAiConfig with aiApiKey / aiProviderPreset) are imported as an AI Provider the first time the GUI starts with a metadata provider, then removed from hop-config.json.