Avro is a data serialization format. Data are described with a schema and compressed in binary.

Stambia can work with Avro.

To accomplish this, you'll first have to install the Stambia's Avro Component and prepare your Designer and environment.

Therefore, you can find below all the explanations and instructions to install and set up everything to get ready to work with Avro.

Prerequisites:
  • Stambia DI Designer S20.4.1 or higher
  • Stambia DI Runtime S20.4.1 or higher

 

Component Installation

Install Avro Component as any other Stambia Component.

Refer to the Stambia DI for Avro article for the download section.

Once the installation is performed, you can continue to the next step.

 

Reverse from an existing file

Avro format use a schema to describe data.

This schema can be in 2 files :

1. Inside a dedicated schema file, formated as a JSON (standard extension .avsc).

Fill the «AVSC Reverse URL» field and reverse them using «Reverse from AVSC» action.

Avro01

2. Inside the binary data file itself (standard extension .avro).

Fill the «AVRO Reverse URL» field and reverse them using «Reverse from AVRO» action.

Avro02

So, Stambia can reverse schema regarding both way.

 

Creation of the metadata

In the Project Explorer, in a project, do a right clic / New / Metadata.

Choose Avro Schema.

Avro03

 

Choose a name for reversed node, then if you have an Avro schema file, fill "AVSC File Path"

Avro05

or you have an Avro data file, fill "AVRO File Path".

Avro04

Then execute reverse and check Stambia designer result match Avro schema.

Beware:reverse can take a few seconds to complete.

Avro Schema can be edited inside Stambia designer, regarding every Avro types :

  • records and fields
  • values (primitive types)
  • fixed values
  • enumerations
  • arrays
  • maps and keys

 

You can find more information about Avro Component in the Designer Help. (Menu Help / Help Contents)

Avro06