0

I'm describing faust model by extending faust.Record class. My input json looks like:

{
  "1": "some_string"
  "2": "some_string"
}

So I have to create class fields, named with numbers only. I know, it restricted in Python in general, but is there any workarounds, like aliases for my fieds names?

I'm looking for something like Jackson annotations in Java:

import faust

class Model(faust.Record, serializer='json'):
    @alias(field_name: '1')
    first_name: str
    @alias(field_name: '2')
    last_name: str
coolsv
  • 268
  • 3
  • 17

1 Answers1

1

There is no simple way to magically salvage a topic without a proper data model.

You should never write to a kafka topic without a data model.

As @OneCricketeer suggested, you should ingest the undefined topic as a dictionary.

After that you can write an Agent that consumes this topic, transforms your model to a proper data model and write it to a topic.

class Person(faust.Record):
    first_name: str
    last_name: str

Now you can start using your e.g. Person model as expected.

undeclared_json_topic = app.topic('name_of_your_input_topic')
person_topic = app.topic('person', value_type=Person)

@app.agent(undeclared_json_topic)
async def migrate_to_person(json_stream: faust.Stream) -> faust.Stream[Person]:
    async for json in json_stream:
        person = Person(first_name=json['1'], last_name=json['2'])
        await person_topic.send(value=person)
        yield
Florian Hall
  • 81
  • 6