Avro

From avro.apache.org:

Apache Avro™ is a data serialization system. To learn more about Avro, please read the current documentation.

Hop supports Avro through a number of plugins. First and foremost we support Avro as a data type called "Avro Record". It simply describes a class of type org.apache.avro.generic.GenericRecord. This in turn encapsulates both an Avro schema and a set of values.

Pipeline Transforms

  • Avro Decode: This allows you to extract Hop values from an Avro Record data type.

  • Avro Encode: This allows you to extract Hop values from an Avro Record data type.

Kafka Producer transforms

If you want to send or receive Avro records using the Hop Kafka transforms it’s important to set a few options:

Option Description Example

value.converter.schema.registry.url

Specify the URL of your Confluent schema registry.

http://confluent-server:8081

schema.registry.url

Specify the URL of your Confluent schema registry.

http://confluent-server:8081

auto.register.schemas

Instruct Kafka to automatically register the schema of the Avro record in the Confluent schema registry.

true