Welcome Guest! Log in
×

Notice

The forum is in read only mode.
Stambia versions 2.x, 3.x, S17, S18, S19 and S20 are reaching End of Support January, 15th, 2024. Please consider upgrading to the supported Semarchy xDI versions. See Global Policy Support and the Semarchy Documentation.

The Stambia User Community is moving to Semarchy! All the applicable resources have already been moved or are currently being moved to their new location. Read more…

Topic-icon Question Sending Avro file using Kafka

More
08 Jun 2021 10:48 #1 by Clément
Sending Avro file using Kafka was created by Clément
Hi ,

I need to send data in Kafka from my database.

I well configured the metadata and I can connect to kafka.

I created the topic and configurated it well .

The schema for the topic is an Avro file with complexe format ( it seems stambia can only handle simple format like key / value )

Schema :

{
"type": "record",
"name": "MeasureCapture",
"namespace": "xxxxxxxxxxxxxxxx",
"doc": "It's a value for a Data ",
"fields": [
{
"name": "measureJobId",
"type": {
"type": "string",
"avro.java.string": "String"
},
"doc": "Uniquely identify a Data Job "
},
{
"name": "dataScopeDateTime",
"type": {
"type": "long",
"logicalType": "timestamp-millis"
},
"doc": "The original data date from your data source "
},
{
"name": "captureDateTime",
"type": {
"type": "long",
"logicalType": "timestamp-millis"
},
"doc": "Timestamp when the measure was captured."
},
{
"name": "result",
"type": "double",
"doc": "Measured value for a specific period (Data Scope Date Time) "
},
{
"name": "distributionKeys",
"type": {
"type": "array",
"items": {
"type": "string",
"avro.java.string": "String"
}
},
"doc": "List of distribution keys of the captured measure to break down results into sub-results. "
}
]
}


Is it possible to send complexe data format in kafka using stambia , is there any Avro metadata that I can use to do this ?

Regards,

Clément DEWEVER
Attachments: