Data Stream Input
Overview
The Data Stream Input transform reads rows from a configured Data Stream into a Hop pipeline.
It supports all available Data Stream implementations (Arrow File Stream, Arrow Random Access File, and Arrow Flight).
Options
| Option | Description |
|---|---|
Data Stream | The name of the source Data Stream to read from. This is a dropdown that shows all defined Data Streams in the project. |
Notes
-
The transform will automatically use the implementation defined in the selected Data Stream.
-
When using Arrow Flight, the transform connects to the running Flight server and pulls data using the stream name.
-
When reading from file-based streams, the file is read sequentially (streaming format) or with random access, depending on the implementation.