Can you put comments in Avro JSON schema files?

Question

I'm writing my first Avro schema, which uses JSON as the schema language. I know you cannot put comments into plain JSON, but I'm wondering if the Avro tool allows comments. E.g. Perhaps it strips them (like a preprocessor) before parsing the JSON.

Edit: I'm using the C++ Avro toolchain

score 16 · Answer 1 · edited Jan 03 '20 at 14:46

Yes, but it is limited. In the schema, Avro data types 'record', 'enum', and 'fixed' allow for a 'doc' field that contains an arbitrary documentation string. For example:

{"type": "record", "name": "test.Weather",
 "doc": "A weather reading.",
 "fields": [
     {"name": "station", "type": "string", "order": "ignore"},
     {"name": "time", "type": "long"},
     {"name": "temp", "type": "int"}
 ]
}

From the official Avro spec:

doc: a JSON string providing documentation to the user of this schema (optional).

https://avro.apache.org/docs/current/spec.html#schema_record

An example: https://github.com/apache/avro/blob/33d495840c896b693b7f37b5ec786ac1acacd3b4/share/test/schemas/weather.avsc#L2

Gaetan E. · Answer 2 · 2013-06-04T08:02:36.890

7

Yes, you can use C comments in an Avro JSON schema : /* something */ or // something
Avro tools ignores these expressions during the parsing.
EDIT: It only works with the Java API.

edited Jun 04 '13 at 08:02

answered May 30 '13 at 11:52

Gaetan E.

401
4
10

1

This doesn't work for me. If I put either style of comments (/**/ or //) into the JSON file, I get an error: "Failed to parse or compile schema: Unexpected character in json 2f". I tried putting both style of comments in several places in the file. – jfritz42 May 31 '13 at 15:48
I suggest that you have ever compiled your schema without the comments and it works. Check the following things : latest version of avro-tools, keep sure that your schema compiles (with the avro tools or [Json validator](http://jsonlint.com/)). For your information, comments work fine for all my avro schema... – Gaetan E. Jun 03 '13 at 08:07
I'm using 1.7.4 which is the latest release. I think the issue might be that we're using the C++ version right now, and you're using the Java version. Sorry to have left out that detail in my question. – jfritz42 Jun 03 '13 at 23:16
1

Indeed... I just tried with the C++ version and it doesn't work. Thanks to highlight this avro issue. – Gaetan E. Jun 04 '13 at 08:01

score 5 · Answer 3 · answered Mar 24 '20 at 07:37

According to the current (1.9.2) Avro specification it's allowed to put in extra attributes, that are not defined, as metadata:

This allows you add comments like this:

{
  "type": "record", 
  "name": "test",
  "comment": "This is a comment",
  "//": "This is also a comment",
  "TODO": "As per this comment we should remember to fix this schema" ,
  "fields" : [
    {
      "name": "a", "type": "long"
    },
    {
      "name": "b", "type": "string"
    }
  ]
}

score 2 · Answer 4 · answered Jan 01 '14 at 20:11

No, it can't in the C++ nor the C# version (as of 1.7.5). If you look at the code they just shove the JSON into the JSON parser without any comment preprocessing - bizarre programming style. Documentation and language support appears to be pretty sloppy...

Can you put comments in Avro JSON schema files?

4 Answers4