Apache Avro "heterogeneous array"

Question

Could someone please advise how to use heterogeneous arrays in Avro (if possible). Cannot figure out what the trick is.

My schema is:

{
  "type": "array", "items": ["null", "string"]
}

I am not using the generated headers, just trying to do it all in code:

avro::ValidSchema schema;
avro::compileJsonSchema(ifs, schema);

avro::EncoderPtr e = avro::binaryEncoder();
avro::DecoderPtr d = avro::binaryDecoder();

avro::GenericDatum datum(schema);
avro::GenericArray& array = datum.value<avro::GenericArray>();

array.value().push_back(avro::GenericDatum(std::string("lala")));

std::auto_ptr<avro::OutputStream> out = avro::memoryOutputStream();
e->init(*out);
avro::encode(*e, datum);

std::auto_ptr<avro::InputStream> in = avro::memoryInputStream(*out);
d->init(*in);

avro::GenericDatum ddatum(schema);
avro::decode(*d, ddatum);

This throwing an exception:

Exception: vector::_M_range_check: __n (which is 4) >= this->size() (which is 2)

I am assuming I am pushing a GenericDatum into the encoded stream, but should somehow specify that that is a value of a Union .. not sure how to do that.

Edit:

I was able to achieve what i want by directly encoding elements of the schema:

std::auto_ptr<avro::OutputStream> out = avro::memoryOutputStream();
e->init(*out);

e->arrayStart();
e->setItemCount(COLS);
char buf[10];
for(int j=0; j < COLS; ++j){
    .....
    e->encodeUnionIndex(1);
    e->encodeString(std::string(buf));
}
e->arrayEnd();
e->flush();

However, if there is a way to do it through GenericDatum is still a mystery to me.

Apache Avro "heterogeneous array"

0 Answers0