Hop Server Web Services Overview

Hop Server has a rich set of web services that can be used to query and manage the server’s status, or to manage pipeline and workflow execution details.

when specified, the id for a workflow or pipeline represents one execution of a workflow or pipeline on the server.

addExport (deprecated)

Deprecated since 2.18.0. Use registerPackage (hop/registerPackage) instead. The remote pipeline and workflow engines call registerPackage for all export operations. This endpoint will be removed in a future release.

name: addExport
description: Upload a resources export file. Add a zipped pipeline or workflow to the body payload as a binary file. Deprecated — use registerPackage instead.
endPoint: GET hop/addExport

The zipped export is sent as the binary request body.

parameters

type: the type of resource in the export, pipeline or workflow
load: the name of the file inside the zip to load and register, e.g. my-workflow.hwf. When left empty the export is only uploaded to the server, nothing is loaded or registered.

example request

http://localhost:8081/hop/addExport/?type=workflow&load=my-workflow.hwf with zipped workflow as payload

result

A zip file with the export is created on the server’s file system. The message contains the url of the loaded file inside that archive, the id contains the id of the registered pipeline or workflow:

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>OK</result>
    <message>zip:file:///tmp/export_70eb8ef1-9721-4cf5-afa3-940cd0f771d9.zip!my-workflow.hwf</message>
    <id>c1451bfb-b867-4c76-b123-c29d2b05da17</id>
</webresult>

When load is left empty the message contains the url of the uploaded archive and the id stays empty.

addPipeline (deprecated)

Deprecated since 2.18.0. Use registerPipeline (hop/registerPipeline) instead. This endpoint is no longer used by the remote pipeline engine and will be removed in a future release.

name

addPipeline

description

Add a pipeline for execution. Deprecated — use registerPipeline instead.

endPoint

GET hop/addPipeline

Content-Type: text/xml;charset=UTF-8

parameters

xml (optional): return the result as xml (default HTML) use &xml=Y

The request body should contain the xml of a pipeline_configuration (a pipeline and a pipeline_execution_configuration wrapped in a pipeline_configuration tag).

example request: http://localhost:8081/hop/addPipeline/?xml=Y with XML payload
result: -

addWorkflow (deprecated)

Deprecated since 2.18.0. Use registerWorkflow (hop/registerWorkflow) instead. This endpoint is no longer used by the remote workflow engine and will be removed in a future release.

name

addWorkflow

description

Add a workflow for execution. Deprecated — use registerWorkflow instead.

endPoint

GET hop/addWorkflow

Content-Type: text/xml;charset=UTF-8

parameters

xml (optional): return the result as xml (default HTML) use &xml=Y

The request body should contain the xml of a workflow_configuration (a workflow and a workflow_execution_configuration wrapped in a workflow_configuration tag).

example request: http://localhost:8081/hop/addWorkflow/?xml=Y with XML payload
result: -

getPipelineImage

name

getPipelineImage

description

Generate a SVG image of a pipeline

endPoint

GET hop/pipelineImage

parameters

name: name of the pipeline to generate the image for
id (required): id of the pipeline to generate the image for

example request

GET http://localhost:8081/hop/pipelineImage/?name=remote-pipeline&id=c1451bfb-b867-4c76-b123-c29d2b05da17

result

an SVG image of the pipeline graph

getPipelineStatus

name

getPipelineStatus

description

Get the status of a pipeline

endPoint

GET hop/pipelineStatus

parameters

name: name of the pipeline to get the status for
id (optional): id of the pipeline to get the status for. When omitted, the first pipeline matching name is used.
xml (optional): return the information as xml (default HTML) use &xml=Y
json (optional): return the information as json (default HTML) use &json=Y
from (optional): the first log line number to return (default 0). Use this to incrementally fetch the log of a running pipeline.
sendResult (optional): include the pipeline result in the response, use &sendResult=Y. Only applies to the xml and json output.

example request

GET http://localhost:8081/hop/pipelineStatus/?name=<NAME>&id=<ID>

result

an HTML response with the execution status, transform details and canvas preview for this pipeline

Status

name

status

description

Get the status of the server

endPoint

GET hop/status

parameters

xml (optional): return the information as xml (default HTML) use &xml=Y
json (optional): return the information as json (default HTML) use &json=Y
useLightTheme (optional): render the HTML status page with the light theme, use &useLightTheme=Y. Only applies to the HTML output.

example request

GET http://localhost:8081/hop/status/

GET http://localhost:8081/hop/status/?json=Y

result: an HTML page with an overview of the pipelines and workflows on the server, their execution details and the server’s configuration details.

getWorkflowImage

name

getWorkflowImage

description

Generate an SVG image of a workflow

endPoint

GET hop/workflowImage

parameters

name: name of the workflow to generate the image for
id (optional): id of the workflow to generate the image for. When omitted, the first workflow matching name is used.

example request

GET http://localhost:8081/hop/workflowImage/?name=<NAME>&id=<ID>

result

an SVG image of the workflow graph

getWorkflowStatus

name

getWorkflowStatus

description

Get the status of a workflow

endPoint

GET hop/workflowStatus

parameters

name: name of the workflow to get the status for. Can be omitted when id is specified.
id (optional): id of the workflow to get the status for. When omitted, the first workflow matching name is used.
xml (optional): return the information as xml (default HTML) use &xml=Y
json (optional): return the information as json (default HTML) use &json=Y
from (optional): the first log line number to return (default 0). Use this to incrementally fetch the log of a running workflow.

example request

GET http://localhost:8081/hop/workflowStatus/?name=<NAME>&id=<ID>

result

an HTML response with the execution status, action details and canvas preview for this workflow

pausePipeline

name

pausePipeline

description

Pause or continue a pipeline

endPoint

GET /hop/pausePipeline

parameters

name: name of the pipeline to pause or restart
id (optional): id of the pipeline to pause or restart. When omitted, the first pipeline matching name is used.
xml (optional): return the result as xml (default HTML) use &xml=Y
json (optional): return the result as json (default HTML) use &json=Y

example request

GET http://localhost:8081/hop/pausePipeline/?name=<NAME>&id=<ID>

result

HTML page with the request status, e.g.

<HTML>

<HEAD>
	<TITLE>Pause pipeline</TITLE>
	<META http-equiv="Refresh" content="2;url=/hop/pipelineStatus?name=<NAME>&id=<ID>">
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H1>Pipeline [tmp] : pause requested.</H1>
	<a href="/hop/pipelineStatus?name=<NAME>&id=<ID>">Back to the pipeline status page</a>
	<p>
		<p>
</BODY>

</HTML>

Prepare Execution

name

prepareExec

description

Prepare the execution of a pipeline

endPoint

GET /hop/prepareExec

parameters

name: the name of the pipeline to prepare execution for
id (optional): the id of the pipeline to prepare execution for. When omitted, the first pipeline matching name is used.
xml (optional): return the result as xml (default HTML) use &xml=Y
json (optional): return the result as json (default HTML) use &json=Y

example request

GET http://localhost:8081/hop/prepareExec/?xml=Y&name=<NAME>&id=<ID>

result

Example result:

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>OK</result>
    <message/>
    <id/>
</webresult>

Register Package

name: registerPackage
description: Upload a resources export file and optionally load and register the pipeline or workflow it contains. This is the endpoint used by the remote pipeline and workflow engines for all export operations, and the replacement for the deprecated addExport.
endPoint: GET /hop/registerPackage

The zipped export is sent as the binary request body.

parameters

type: the type of resource to load from the export, pipeline or workflow
load: the name of the file inside the zip to load, e.g. my-pipeline.hpl. When left empty the export is only uploaded to the server, nothing is loaded or registered and the response contains only the xml header, without a webresult.

this endpoint always responds with xml, the xml and json parameters have no effect.

example request: http://localhost:8081/hop/registerPackage/?type=pipeline&load=my-pipeline.hpl with the zipped export as payload
result: The url of the loaded file inside the uploaded archive as the message, and the id of the registered pipeline or workflow as the id:

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>OK</result>
    <message>zip:file:///tmp/export_70eb8ef1-9721-4cf5-afa3-940cd0f771d9.zip!my-pipeline.hpl</message>
    <id>c1451bfb-b867-4c76-b123-c29d2b05da17</id>
</webresult>

Register Pipeline

name

registerPipeline

description

endPoint

GET hop/registerPipeline

Content-Type: text/xml;charset=UTF-8

parameters

xml (optional): return the result as xml (default HTML) use &xml=Y
json (optional): return the result as json (default HTML) use &json=Y

The request body should contain the xml of a pipeline_configuration (a pipeline and a pipeline_execution_configuration wrapped in a pipeline_configuration tag).

example request: http://localhost:8081/hop/registerPipeline/?xml=Y

with XML payload (example):

<pipeline_configuration>
<pipeline>
  <info>
    <name>generate_rows</name>
    <name_sync_with_filename>Y</name_sync_with_filename>
    <description/>
    <extended_description/>
    <pipeline_version/>
    <pipeline_type>Normal</pipeline_type>
    <parameters>
    </parameters>
    <capture_transform_performance>N</capture_transform_performance>
    <transform_performance_capturing_delay>1000</transform_performance_capturing_delay>
    <transform_performance_capturing_size_limit>100</transform_performance_capturing_size_limit>
    <created_user>-</created_user>
    <created_date>2022/02/03 13:47:49.645</created_date>
    <modified_user>-</modified_user>
    <modified_date>2022/02/03 13:47:49.645</modified_date>
    <key_for_session_key>H4sIAAAAAAAA/wMAAAAAAAAAAAA=</key_for_session_key>
    <is_key_private>N</is_key_private>
  </info>
  <notepads>
  </notepads>
  <order>
    <hop>
      <from>Generate rows</from>
      <to>Delay row</to>
      <enabled>Y</enabled>
    </hop>
    <hop>
      <from>Delay row</from>
      <to>result</to>
      <enabled>Y</enabled>
    </hop>
  </order>
  <transform>
    <name>Delay row</name>
    <type>Delay</type>
    <description/>
    <distribute>Y</distribute>
    <custom_distribution/>
    <copies>1</copies>
    <partitioning>
      <method>none</method>
      <schema_name/>
    </partitioning>
    <scaletime>seconds</scaletime>
    <timeout>1</timeout>
    <attributes/>
    <GUI>
      <xloc>416</xloc>
      <yloc>96</yloc>
    </GUI>
  </transform>
  <transform>
    <name>Generate rows</name>
    <type>RowGenerator</type>
    <description/>
    <distribute>Y</distribute>
    <custom_distribution/>
    <copies>1</copies>
    <partitioning>
      <method>none</method>
      <schema_name/>
    </partitioning>
    <fields>
      <field>
        <length>-1</length>
        <name>value</name>
        <precision>-1</precision>
        <set_empty_string>N</set_empty_string>
        <type>String</type>
        <nullif>test</nullif>
      </field>
    </fields>
    <interval_in_ms>5000</interval_in_ms>
    <last_time_field>FiveSecondsAgo</last_time_field>
    <never_ending>N</never_ending>
    <limit>10000</limit>
    <row_time_field>now</row_time_field>
    <attributes/>
    <GUI>
      <xloc>160</xloc>
      <yloc>96</yloc>
    </GUI>
  </transform>
  <transform>
    <name>result</name>
    <type>Dummy</type>
    <description/>
    <distribute>Y</distribute>
    <custom_distribution/>
    <copies>1</copies>
    <partitioning>
      <method>none</method>
      <schema_name/>
    </partitioning>
    <attributes/>
    <GUI>
      <xloc>720</xloc>
      <yloc>96</yloc>
    </GUI>
  </transform>
  <transform_error_handling>
  </transform_error_handling>
  <attributes/>
</pipeline>
  <pipeline_execution_configuration>
    <pass_export>N</pass_export>
    <parameters>
    </parameters>
    <variables>
    <variable><name>HOP_AUDIT_FOLDER</name><value>/Users/hans/config/audit</value></variable>
    <variable><name>HOP_AUTO_CREATE_CONFIG</name><value>Y</value></variable>
    <variable><name>HOP_CONFIG_FOLDER</name><value>/Users/hans/config</value></variable>
    <variable><name>HOP_DATASETS_FOLDER</name><value>/Users/hans/test/datasets</value></variable>
    <variable><name>HOP_ENVIRONMENT_NAME</name><value>test1</value></variable>
    <variable><name>HOP_METADATA_FOLDER</name><value>/Users/hans/test/metadata</value></variable>
    <variable><name>HOP_PIPELINE_PAN_JVM_EXIT_CODE</name><value/></variable>
    <variable><name>HOP_PROJECTS</name><value>/Users/hans/tmp/</value></variable>
    <variable><name>HOP_PROJECT_NAME</name><value>test</value></variable>
    <variable><name>HOP_UNIT_TESTS_FOLDER</name><value>/Users/hans/test</value></variable>
    <variable><name>NEO4J_CONNECTION</name><value>neo4j</value></variable>
    <variable><name>PROJECT_HOME</name><value>/Users/hans/test</value></variable>
    <variable><name>TEST</name><value>TEST</value></variable>
    <variable><name>ftp.nonProxyHosts</name><value>local|*.local|169.254/16|*.169.254/16</value></variable>
    <variable><name>http.nonProxyHosts</name><value>local|*.local|169.254/16|*.169.254/16</value></variable>
    <variable><name>jdk.debug</name><value>release</value></variable>
    <variable><name>native.encoding</name><value>UTF-8</value></variable>
    <variable><name>p1</name><value>a</value></variable>
    <variable><name>socksNonProxyHosts</name><value>local|*.local|169.254/16|*.169.254/16</value></variable>
    </variables>
    <log_level>Basic</log_level>
    <log_file>N</log_file>
    <log_filename/>
    <log_file_append>N</log_file_append>
    <create_parent_folder>N</create_parent_folder>
    <clear_log>Y</clear_log>
    <show_subcomponents>Y</show_subcomponents>
    <run_configuration>local</run_configuration>
</pipeline_execution_configuration>
<metastore_json>H4sIAAAAAAAA/+1XbVPbOBD+Kxl/upshzQsUAt/SxBzchSRNTDudpuNRZNlRkSVXkgk5hv9+K0t+
CSl3cDP37coH8Gq1evTs7rPqo6eIvCfSu/j66G3JephlU5QS78Lzjjyl2EjwmCbeBc8ZKww3IoLV
GDFFjrxMiofdXEht/TdCaW539/pn77rw0wNzhpTaChmB2edY7jJNolZ/Tc4HKMaDU4TO4j45wese
Qv3T9zgyJjyAjVzwuTnhCuIqe4QArJJGxH+gSlOewHpGpKZENUE50w1Smkh7H3uBzGId9AYGmMPK
ydZzd7mqbwCmHLhxX5jlJpb39A08aUYY5aQNW9aw+BVsOae6rYnS9lNG61QVnLq/Hr3ldPb5cjL8
wzcfEdJojRSpuc5YnlB+bUiqPY88yiPyEKA1IypDuLrHFkmyEYDPbjbhDp0QxkSpYJeBpbuXnZeS
4lUMVYgcwiUX25ihO2LKoqgY3qAVaS3pOtfE3vR2Pp8tgmUYXN/4y2B4Mw/Hw2AYBl/mcHnvC4T4
eDsL/HA4mYSX1/5kvATzFMzVzg+z2cQfTg/2Xc4WIz+8HvvT4Bp2LuCMWTiZffYXo+HSd1HmC3/p
Lz75ofs9Dj/PFuPQeZgwy4+TcDSbTv1RYO/607gA5nncS3+xgHjL0ZV/Mwynwxuz5j0deSniOWK3
kh1Ujvf0VFVaQjiRFPdhR6My5rNl8Btg/Tj5SWlkkLVEArF7JdLY8mKNvLUomMCIme83tOzZOSa9
bnR6fox6+Lwun/cnx/3nJTS3N7GY/6+ht9TQBnHVrKMsKYQIMnQXM7FtM5FY3cGQN8QjidpYcE6w
poLbFU7Eyfd2IlG2aaeg4cyaQfLbJhfUVAjIlZZwVCxk6pI2ztN01/olEi0u9AYE91dAFlNGHJJV
5xa2q1XHQFx1jAKuOqbyljbou03Gih2ERS7kPWK5kRFmFXypkc5r9XZxFUozRkKAV6KDywmuCde2
dj2UZYxiZG646nxXcM8jD1BBvUPVapmTPamWOTeUwCzLJXKsPII/lCdZ5Hy0v/TofSAoHVMJDM5d
CL/wNWuapNlEYOdbsHGx6pifeyRXnViwqKDkvgu0fE9P77Ik6nbPTjmO7rJ4IJPjaNuFfwkAD1Yd
0w1awnlAxpXIgjID6kMex0Qu6Z9OsnmerokMRRyaxMMR1hwj/TuSdc0MQWVMD0KsF0NfwjjbXAOd
ErJht1aZnxdNO2JQSkQ1R4EKBCQrcWgesgNHU6F7JH9CkhYK5GrQ5rYYk6BPRGFJM8diIYkv52NB
UqEL9smD0ZgQJFHkEpvQJtkwcktDqJFMiA5tHizYetH+3lu0WhRmgrGQ7nECVRNi9wKy+li8crKw
fDS5t0MzArQW2v0jG49VK/cOqThyPQJFXvS5c5WWgzcyNylgwx8xIdEa4btQ2Yp6b2rQgLet5qy9
pk1DqwElYZJTWJqgIm9SbBXQW/u7HTEJ0+bjUG3ENiwPraxaZAKSYKaEsyRIb4C8lID+41oIinpE
hYIZ/IX1lQVWZmqfJyBHtXaQ/lapCqpVeLJda0v1pgUwWkqDfCIZ2ZUWNE3l3bIc24wUarqnsY/V
S3pvkq4LuqR7JJ+dnJ2491pjxhfRvMYAcNdBuRYp3BOX1yxnZa6god1gLk53nEmRG0mtn8KvHuPr
qH8cRYNujAupfXZAfb5XwHwI6uQsIG+7gJbvypqTCb2H545Sow3Bd8YBwD33ecE8xD9yquiBB3S6
0kPGRuZ9Hxv1J+oA26iBICa6QubI2fevKH7O4VzAdNlZp7Vgukzg6eDsBSBVBCgCBaeflMrEn2e5
AXEuBKsVvjHyS+8yWBO2qcBU8ERE64M5nyFAVH8iteO4vT/lm3OxejlUb4m3TspKXw4V4KUWbrZp
/2/6tMT0uj4tvas+fZ0s/jvY/x3qb/Z91Y5ITDltJAAzoXJZvgWa/3U9GB/Fc2tcBXByoggUB9JC
ljWkYK7zO7toJAlk3Xw8/QVlC+SCFRAAAA==
</metastore_json>
</pipeline_configuration>

The metastore_json is base64 encoded GZip content

result: -

Register Workflow

name

registerWorkflow

description

endPoint

GET /hop/registerWorkflow

Content-Type: text/xml;charset=UTF-8

parameters

xml (optional): return the result as xml (default HTML) use &xml=Y
json (optional): return the result as json (default HTML) use &json=Y

The request body should contain the xml of a workflow_configuration (a workflow and a workflow_execution_configuration wrapped in a workflow_configuration tag), in the same form as the registerPipeline payload.

example request: http://localhost:8081/hop/registerWorkflow/?xml=Y with XML payload
result: The id of the registered workflow, e.g.

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>OK</result>
    <message>Workflow 'my-workflow' was added to the list with id c1451bfb-b867-4c76-b123-c29d2b05da17</message>
    <id>c1451bfb-b867-4c76-b123-c29d2b05da17</id>
</webresult>

Remove Pipeline

name

removePipeline

description

Remove a pipeline from the server

endPoint

GET /hop/removePipeline

parameters

name: the name of the pipeline to remove
id (optional): the id of the pipeline to remove. When omitted, the first pipeline matching name is used.
xml (optional): return the result as xml (default HTML) use &xml=Y
json (optional): return the result as json (default HTML) use &json=Y

example request

GET http://localhost:8081/hop/removePipeline/?name=<NAME>&id=<ID>

result

Example result:

<HTML>

<HEAD>
	<TITLE>The pipeline was removed</TITLE>
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H3>The pipeline with name [<NAME>] and hopServer object id <ID> was removed from Hop
		Server.</H3>
	<a href="/hop/status">Back to the status page</a><br>
	<p>
</BODY>

Remove Workflow

name

removeWorkflow

description

Remove a workflow from the server

endPoint

GET /hop/removeWorkflow

parameters

name: the name of the workflow to remove
id (optional): the id of the workflow to remove. When omitted, the first workflow matching name is used.
xml (optional): return the result as xml (default HTML) use &xml=Y
json (optional): return the result as json (default HTML) use &json=Y

example request

GET http://localhost:8081/hop/removeWorkflow/?name=<NAME>&id=<ID>

result

Example result:

<HTML>

<HEAD>
	<TITLE>The workflow was removed</TITLE>
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H3>The workflow with name [<NAME>] and hopServer object id <ID> was removed from Hop
		Server.</H3>
	<a href="/hop/status">Back to the status page</a><br>
	<p>
</BODY>

Sniff Transform

name

sniffTransform

description

Sniff test a pipeline transform

endPoint

GET /hop/sniffTransform

parameters

pipeline: name of the pipeline to sniff
id (optional): id of the pipeline to sniff. When omitted, the first pipeline matching pipeline is used.
transform: name of the transform to sniff
type (optional): sniff input or output (default output)
lines (optional): number of lines to wait for (default 0)
copynr (optional): when using multiple copies you can specify the copynr (default 0)
xml (optional): return the rows as xml (default HTML) use &xml=Y
json (optional): return the rows as json (default HTML) use &json=Y

following parameters are required

id or pipeline
transform

example request

GET http://localhost:8081/hop/sniffTransform/?xml=Y&pipeline=<NAME>&id=<ID>&transform=<TRANSFORMNAME>&type=output&lines=100

result

Example result (empty):

<row-buffer>
<row-meta/>
</row-buffer>

Start Pipeline Execution

name

startExec

description

Start the execution of a pipeline

endPoint

GET /hop/startExec

parameters

name: the name of the pipeline to start
id (optional): the id of the pipeline to start. When omitted, the first pipeline matching name is used.
xml (optional): return the result as xml (default HTML) use &xml=Y
json (optional): return the result as json (default HTML) use &json=Y

example request

GET http://localhost:8081/hop/startExec?name=<NAME>&id=<ID>

result

Example result:

<HTML>

<HEAD>
	<TITLE>Prepare execution of pipeline</TITLE>
	<META http-equiv="Refresh" content="2;url=/hop/status?name=<NAME>">
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H1>Pipeline '<NAME>' has been executed.</H1>
	<a href="/hop/pipelineStatus?name=<NAME>&id=8bea27db-de97-4bd0-a210-d9bba3aacac2">Back to the pipeline status page</a>
	<p>
		<p>
</BODY>

</HTML>

Start Pipeline

name

startPipeline

description

Prepare and start the execution of a pipeline

endPoint

GET /hop/startPipeline

parameters

name: the name of the pipeline to start
id (optional): the id of the pipeline to start. When omitted, the first pipeline matching name is used.
xml (optional): return the result as xml (default HTML) use &xml=Y
json (optional): return the result as json (default HTML) use &json=Y

example request

GET http://localhost:8081/hop/startPipeline?name=<NAME>&id=<ID>

result

Example result:

<HTML>

<HEAD>
	<TITLE>Start of pipeline</TITLE>
	<META http-equiv="Refresh" content="2;url=/hop/pipelineStatus?name=<NAME>">
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H1>Pipeline [<NAME>] was started.</H1>
	<a href="/hop/pipelineStatus?name=<NAME>&id=8bea27db-de97-4bd0-a210-d9bba3aacac2">Back to the status page</a>
	<p>
		<p>
</BODY>

</HTML>

Start Workflow

name

startWorkflow

description

Prepare and start the execution of a workflow

endPoint

GET /hop/startWorkflow

parameters

name: the name of the workflow to start
id (optional): the id of the workflow to start. When omitted, the first workflow matching name is used.
xml (optional): return the result as xml (default HTML) use &xml=Y
json (optional): return the result as json (default HTML) use &json=Y

example request

GET http://localhost:8081/hop/startWorkflow?name=<NAME>&id=<ID>

result

Example result:

<HTML>

<HEAD>
	<TITLE>Start workflow</TITLE>
	<META http-equiv="Refresh" content="2;url=/hop/status?name=<NAME>">
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H1>Workflow [<NAME>] was started.</H1>
	<a href="/hop/workflowStatus?name=<NAME>&id=8bea27db-de97-4bd0-a210-d9bba3aacac2">Back to the status page</a>
	<p>
		<p>
</BODY>

</HTML>

Execute Pipeline from File

name

execPipeline

description

Execute a pipeline directly from a file path on the server’s file system. The pipeline is loaded, executed synchronously, and the result is returned immediately. This servlet supports variable resolution in file paths (e.g., ${PROJECT_HOME}) and can pass parameters and variables to the pipeline.

endPoint

GET /hop/execPipeline

parameters

pipeline (required): File path to the pipeline (.hpl file). Can use variables like ${PROJECT_HOME} which will be resolved by the server.
level (optional): Logging level (e.g., Debug, Basic, Detailed, Error, Minimal, Nothing, RowLevel). Defaults to server’s default log level.
runConfig (optional): Run configuration name to use for execution. If not specified, the default run configuration from metadata is used, or "local" if no default is found.
json (optional): return the result as json (default xml) use &json=Y. Note that, unlike pipeline, level and runConfig, this parameter is also passed to the pipeline as a variable.
any parameter name: Any additional URL parameters will be passed as pipeline parameters or variables. If the parameter name matches a pipeline parameter, it will be set as a parameter; otherwise, it will be set as a variable.

example request

GET http://localhost:8081/hop/execPipeline?pipeline=/opt/hop/config/projects/samples/pipelines/pipeline-with-parameter.hpl&PRM_EXAMPLE=test_value&level=BASIC

Using project variables: GET http://localhost:8081/hop/execPipeline?pipeline=${PROJECT_HOME}/pipelines/my-pipeline.hpl&runConfig=local&level=Detailed

result: Success response (HTTP 200):

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>OK</result>
    <message>Pipeline executed successfully</message>
    <id/>
</webresult>

Error response - Missing parameter (HTTP 400):

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>ERROR</result>
    <message>Missing mandatory parameter: pipeline</message>
    <id/>
</webresult>

Error response - File not found (HTTP 404):

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>ERROR</result>
    <message>Unable to find pipeline: /path/to/nonexistent.hpl (resolved: /path/to/nonexistent.hpl)</message>
    <id/>
</webresult>

Error response - Execution error (HTTP 500):

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>ERROR</result>
    <message>Error executing pipeline: [log output]</message>
    <id/>
</webresult>

notes

The pipeline path supports variable resolution. If the server is started with a project (using -j flag), variables like ${PROJECT_HOME} will be resolved.
The execution is synchronous - the servlet waits for the pipeline to complete before returning.
The pipeline is automatically registered in the server’s pipeline map and can be queried using getPipelineStatus.
All URL parameters except pipeline, level, and runConfig are passed to the pipeline as parameters or variables.

Execute Workflow from File

name

execWorkflow

description

Execute a workflow directly from a file path on the server’s file system. The workflow is loaded, executed synchronously, and the result is returned immediately. This servlet supports variable resolution in file paths (e.g., ${PROJECT_HOME}) and can pass parameters and variables to the workflow.

endPoint

GET /hop/execWorkflow

parameters

workflow (required): File path to the workflow (.hwf file). Can use variables like ${PROJECT_HOME} which will be resolved by the server.
level (optional): Logging level (e.g., Debug, Basic, Detailed, Error, Minimal, Nothing, RowLevel). Defaults to server’s default log level.
runConfig (optional): Run configuration name to use for execution. If not specified, the default run configuration from metadata is used, or "local" if no default is found.
json (optional): return the result as json (default xml) use &json=Y. Note that, unlike workflow, level and runConfig, this parameter is also passed to the workflow as a variable.
any parameter name: Any additional URL parameters will be passed as workflow parameters or variables. If the parameter name matches a workflow parameter, it will be set as a parameter; otherwise, it will be set as a variable.

example request

GET http://localhost:8081/hop/execWorkflow?workflow=/opt/hop/config/projects/samples/workflows/parallel/parallel-workflow.hwf&level=BASIC

Using project variables: GET http://localhost:8081/hop/execWorkflow?workflow=${PROJECT_HOME}/workflows/my-workflow.hwf&runConfig=local&level=Detailed

result: Success response (HTTP 200):

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>OK</result>
    <message>Workflow executed successfully</message>
    <id/>
</webresult>

Error response - Missing parameter (HTTP 400):

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>ERROR</result>
    <message>Missing mandatory parameter: workflow</message>
    <id/>
</webresult>

Error response - File not found (HTTP 404):

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>ERROR</result>
    <message>Unable to find workflow: /path/to/nonexistent.hwf (resolved: /path/to/nonexistent.hwf)</message>
    <id/>
</webresult>

Error response - Execution error (HTTP 500):

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>ERROR</result>
    <message>Error executing workflow: [log output]</message>
    <id/>
</webresult>

notes

The workflow path supports variable resolution. If the server is started with a project (using -j flag), variables like ${PROJECT_HOME} will be resolved.
The execution is synchronous - the servlet waits for the workflow to complete before returning.
The workflow is automatically registered in the server’s workflow map and can be queried using getWorkflowStatus.
All URL parameters except workflow, level, and runConfig are passed to the workflow as parameters or variables.

Register Execution Information

name

registerExecInfo

description

Register execution information at an execution information location defined in the server’s metadata. This endpoint is used by the pipeline and workflow engines to report execution information to a central location. The payload to register is sent as the request body.

endPoint

GET /hop/registerExecInfo

parameters

type: the type of execution information in the body. One of execution, state, data.
location: the name of the execution information location (as defined in the metadata) to register the information at.

example request

GET http://localhost:8081/hop/registerExecInfo/?type=execution&location=<LOCATION_NAME> with the execution information json as payload

result

an xml response confirming the registration:

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>OK</result>
    <message>Registration successful at location &lt;LOCATION_NAME&gt;</message>
    <id/>
</webresult>

Get Execution Information

name

getExecInfo

description

Query an execution information location defined in the server’s metadata. The type parameter selects the kind of query to perform, and determines which of the other parameters are needed.

endPoint

GET /hop/getExecInfo

parameters

type: the type of query to perform, see the table below.
location: the name of the execution information location (as defined in the metadata) to query.
id: the execution id to query. Required for types STATE, STATE_LOGGING, EXECUTION, CHILDREN, CHILD_IDS, PARENT_ID and DELETE, optional for DATA.
parentId: the parent execution id. Required for type DATA.
name: the name of the pipeline or workflow. Required for type LAST_EXECUTION.
execType: the execution type, one of Pipeline, Workflow, Transform or Action. Required for types LAST_EXECUTION and CHILD_IDS.
children (optional): include child executions, use &children=Y. Only applies to type IDS.
limit (optional): for type IDS the maximum number of ids to return (default 100), for type STATE_LOGGING the maximum size of the logging text to return (default -1, no limit).

The type parameter accepts the following values:

type description required parameters

type	description	required parameters
`STATE`	Get the execution state of an execution	id
`STATE_LOGGING`	Get the logging text of an execution state	id
`IDS`	Get a list of execution ids	-
`EXECUTION`	Get a single execution	id
`CHILDREN`	Get the child executions of an execution	id
`DATA`	Get the execution data of an execution	parentId
`LAST_EXECUTION`	Get the last execution with a given name and type	name, execType
`CHILD_IDS`	Get the child ids of an execution	id, execType
`PARENT_ID`	Get the parent id of a child execution	id
`DELETE`	Delete an execution	id

STATE

Get the execution state of an execution

STATE_LOGGING

Get the logging text of an execution state

IDS

Get a list of execution ids

EXECUTION

Get a single execution

CHILDREN

Get the child executions of an execution

DATA

Get the execution data of an execution

parentId

LAST_EXECUTION

Get the last execution with a given name and type

name, execType

CHILD_IDS

Get the child ids of an execution

id, execType

PARENT_ID

Get the parent id of a child execution

DELETE

Delete an execution

the type and execType values are case sensitive. Use type=STATE_LOGGING and execType=Pipeline exactly as written above, anything else results in an error.

example request: GET http://localhost:8081/hop/getExecInfo/?type=IDS&location=<LOCATION_NAME>&limit=10

GET http://localhost:8081/hop/getExecInfo/?type=STATE&location=<LOCATION_NAME>&id=<ID>

result: a json response with the requested execution information. This endpoint always responds with json.

Delete Execution Information (deprecated)

Deprecated since 2.18.0. Use getExecInfo (hop/getExecInfo) with type=DELETE instead. This endpoint has no callers in the application and will be removed in a future release.

name

deleteExecInfo

description

Delete an execution from an execution information location defined in the server’s metadata. Deprecated — use getExecInfo with type=DELETE instead.

endPoint

GET /hop/deleteExecInfo

parameters

id: the id of the execution to delete.
location: the name of the execution information location (as defined in the metadata) to delete the execution from.

example request

GET http://localhost:8081/hop/deleteExecInfo/?id=<ID>&location=<LOCATION_NAME>

result

an xml response confirming the deletion:

<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>OK</result>
    <message>Execution deletion was successful at location &lt;LOCATION_NAME&gt;</message>
    <id/>
</webresult>

Stop Pipeline

name

stopPipeline

description

Stop a pipeline

endPoint

GET /hop/stopPipeline

parameters

name: the name of the pipeline to stop
id (optional): the id of the pipeline to stop. When omitted, the first pipeline matching name is used.
xml (optional): return the result as xml (default HTML) use &xml=Y
json (optional): return the result as json (default HTML) use &json=Y

example request

GET http://localhost:8081/hop/stopPipeline?name=<NAME>&id=<ID>

result

Example Result:

<HTML>

<HEAD>
	<TITLE>Stop pipeline</TITLE>
	<META http-equiv="Refresh" content="2;url=/hop/pipelineStatus?name=<NAME>">
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H1>Pipeline [<NAME>] stop requested.</H1>
	<a href="/hop/pipelineStatus?name=<NAME>&id=<ID>">Back to the pipeline status page</a>
	<p>
		<p>
</BODY>

</HTML>

Stop Workflow

name

stopWorkflow

description

Stop a workflow

endPoint

GET /hop/stopWorkflow

parameters

name: the name of the workflow to stop
id (optional): the id of the workflow to stop. When omitted, the first workflow matching name is used.
xml (optional): return the result as xml (default HTML) use &xml=Y
json (optional): return the result as json (default HTML) use &json=Y

example request

GET http://localhost:8081/hop/stopWorkflow?name=<NAME>&id=<ID>

result

Example Result:

<HTML>

<HEAD>
	<TITLE>Stop workflow</TITLE>
	<META http-equiv="Refresh" content="2;url=/hop/workflowStatus?name=<NAME>">
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H1>Workflow [<NAME>] stop requested.</H1>
	<a href="/hop/workflowStatus?name=<NAME>&id=<ID>">Back to the pipeline status page</a>
	<p>
		<p>
</BODY>

</HTML>