Create Avro schema with multiple decimal logical data types which have different precision and scale

Question

I need to write several attributes from a java object as fields into Avro schema record including decimal values with different precision and scale. (Question regarding Avro 1.11.1)

Now according to the Avro documentation I should be able annotate a primitive bytes type as decimal logical data type with defined precision and scale. This is the result what I would like to achieve:

{
  "type" : "record",
  "name" : "Object_1",
  "namespace" : "test_namespace",
  "fields" : [ {
    "name" : "first_name",
    "type" : [ "null", "string" ],
    "default" : null
  }, {
    "name" : "last_name",
    "type" : [ "null", "string" ],
    "default" : null
  }, {
    "name" : "salary1",
    "type" : [ "null", {
      "type" : "bytes",
      "logicalType" : "decimal",
      "precision" : 30,
      "scale" : 10
    } ],
    "default" : null
  }, {
    "name" : "salary2",
    "type" : [ "null", {
      "type" : "bytes",
      "logicalType" : "decimal",
      "precision" : 20,
      "scale" : 8
    } ],
    "default" : null
  }]
}

Notice the salary1 and salary2, both are decimal but have different precision and scale property.

Here is the java code for adding of the bytes/decimal field to the schema (its simplified):

    private FieldAssembler<Schema> addAttributeTypeDecimal(final TypeBuilder<Schema> sb, final FieldAssembler<Schema> fa,
                    final boolean isAttributeNullable, final int precision, final int scale) {
        final var schema = LogicalTypes.decimal(precision, scale)).addToSchema(sb.bytesType());
        if (isAttributeNullable)
            return fa.name(drioa.getName()).type().optional().type(schema);
        else
            return fa.name(drioa.getName()).type(schema).noDefault();
    }

The above code is failing if I try to add salary2 using the addToSchema function with exception:

org.apache.avro.AvroRuntimeException: Can't overwrite property: scale
    at org.apache.avro.JsonProperties.addProp(JsonProperties.java:288)
    at org.apache.avro.JsonProperties.addProp(JsonProperties.java:261)
    at org.apache.avro.Schema.addProp(Schema.java:189)
    at org.apache.avro.LogicalTypes$Decimal.addToSchema(LogicalTypes.java:285)

Can you please suggest what is wrong in my logic? From debugging I understood that it is trying to overwrite the bytes decimal data logic type in the list of primitive types, which is not allowed. But how can I prevent it from overwriting it and instead to store a another decimal type specification with different scale and precision?

I saw that there was a similar question asked by Eoin O'Hehir but without any response.

Create Avro schema with multiple decimal logical data types which have different precision and scale

0 Answers0