Apache Arrow Random Access File Data Stream
Overview
This implementation writes/reads the official Arrow IPC File Format (with footer). It supports random access and is ideal when you need to read the same file multiple times or seek within the data.
Configuration
| Property | Description |
|---|---|
Name |
Unique name of the Data Stream |
File name |
Path to the |
Batch size |
Rows per batch when writing (default 10,000) |
Compression |
None, LZ4, Zstd |
When to use
-
You want to archive or share the data
-
Downstream processes need random access
-
You prefer the standard Arrow File format over streaming