2

I created a topic where I store change events of an customer entity. To process the content of that topic, I created a stream with

CREATE STREAM customerstream(event VARCHAR,content MAP<STRING,STRING>) WITH (KAFKA_TOPIC='mycustomers',VALUE_FO
RMAT='JSON');

The content of the stream looks like the following

|EVENT                                                   |CONTENT                                                 |
+--------------------------------------------------------+--------------------------------------------------------+
|create                                                  |{name=bob, location=NY, id=1}                           |
|update                                                  |{location=AM, id=1}                                           |
|update                                                  |{location=BER, id=1}                                          |
|update                                                  |{name=bob_new, id=1}                                          |

As you can see, the topic consists of simple create, delete and update events. I could now write a client to re-construct the entity out of the events. However, I am wondering, if there is a smarter way to do this in ksql so that I do not have to write a client.

The result should look like this

{
name:'bob_new',
location:'BER',
id:1
}

How would the client look when also delete and re-create events would exist?

EDIT:

With updates only the following query works. But it fails if an entity is deleted and later on re-created with different values.

select content['id'],latest_by_offset(content['location'],latest_by_offset(content['name'])) from customerstrea
m2 group by content['id'] emit changes;

Example:

|EVENT                                                   |CONTENT                                                 |
+--------------------------------------------------------+--------------------------------------------------------+
|create                                                  |{name=bob, location=NY, id=1}                           |
|update                                                  |{location=AM, id=1}                                           |
|delete                                                  |{id=1}                                          |
|create                                                  |{name=new_perosn, id=1}                                          |
user3579222
  • 1,103
  • 11
  • 28
  • Deletions + re-creation with same id and different values is suspicious from a DDD standpoint. DDD tells us the entity should match some thing in the real world => deletion/re-creation with other value means that thing somehow vanished and then spawned back in a different state, with no events in between, does that really happen IRL? If the deletion/recreation is done due to the back-end re-using an old id for some new thing, then this contradicts event-sourcing because the history of the old thing is still recorded in the old events, and get stitched together with the history of the new one – Svend Jul 11 '22 at 06:27
  • Anyway: If I want to filter out the delted persons, how can I do that? A stream-stream join requires a WITHIN clause which does not make sense here... – user3579222 Jul 12 '22 at 06:39

0 Answers0