Avro
From avro.apache.org:
Apache Avro™ is a data serialization system. To learn more about Avro, please read the current documentation.
Hop supports Avro through a number of plugins.
First and foremost we support Avro as a data type called "Avro Record". It simply describes a class of type org.apache.avro.generic.GenericRecord. This in turn encapsulates both an Avro schema and a set of values.
Pipeline Transforms
-
Avro decode: This allows you to extract Hop values from an Avro Record data type.
-
Avro encode: This allows you to build an Avro Record from a selection of Hop fields.
-
Avro file input: This allows you to read Avro records from one or more files.
-
Avro file output: This allows you to write Avro records to a file or a field.
Kafka producer transforms
If you want to send or receive Avro records using the Hop Kafka transforms it’s important to set a few options:
| Option | Description | Example |
|---|---|---|
|
Specify the URL of your Confluent schema registry. |
|
|
Specify the URL of your Confluent schema registry. |
|
|
Instruct Kafka to automatically register the schema of the Avro record in the Confluent schema registry. |
true |