(EXPERIMENTAL) Beam Hive Catalog Input

Description

The Beam Hive Catalog input transform reads data from an Apache Hive metastore.

This transform is in an experimental state, backwards compatibility between versions is not guaranteed

Supported Engines

Hop Engine

Not Supported

Spark

Supported

Flink

Maybe Supported

Dataflow

Maybe Supported

Options

Option Description

Option	Description
Transform name	Name of the transform, this name has to be unique in a single pipeline.
Hive Metastore URI	The Thrift metastore URI needs to be in format `thrift://hivemetastore:9083`
Database name	Database name used in the metastore.
Table name	Table name used in the metastore

Transform name

Name of the transform, this name has to be unique in a single pipeline.

Hive Metastore URI

The Thrift metastore URI needs to be in format thrift://hivemetastore:9083

Database name

Database name used in the metastore.

Table name

Table name used in the metastore

Limitations

At this point in time this transform is still in an experimental phase.
It can not yet detect and cast field types, all data is returned in a single string which is separated by the ; character. Use a split fields transform to split the data into fields.