0

There is an event and I want to make sure it delivered.

You know, in the outbox pattern, we save the event to the db before sending it to the message broker and delete it after publishing.

However, if I apply it in inbox pattern, we first read the message from the broker on the consumer side and delete it after the transaction.

But my problem is, there are 100 pending events on the message broker right now, and just in the meantime, the message broker crashed and the events disappeared. These events are not in my Outbox table because I deleted them when it was published. It is not in my inbox table because the consumer has not read it yet.

How can I solve this situation?

Software Engineer
  • 15,457
  • 7
  • 74
  • 102
doğukan güney
  • 1
  • Please clarify. You can't delete specific records in Kafka. They don't disappear from Kafka or RabbitMQ when broker crashes either – OneCricketeer Dec 15 '22 at 07:05
  • Removed specific technology tags since problem is not unique to either. – OneCricketeer Dec 15 '22 at 07:06
  • Deleting from the db should be done by a handler listening for events on the bus. Record the time the record was written along with the record, that way, you can have a separate job that retries a message if it hasn't been deleted within a given timeframe. – Software Engineer Dec 15 '22 at 07:06
  • Deleting from the db cannot be done after the consumer end of the process because there are more than one consumer and more than one consumer can read an event at the same time and this will be an error. By the way, what the message broker is doesn't matter here. Maybe not deleted in Kafka, but deleted in rabbitmq. The issue that I draw attention to here is that these two patterns do not actually make the event arrive. – doğukan güney Dec 15 '22 at 07:12

0 Answers0