1

I am currently trying to get into microservices architecture, and I came across Data consistency issue. I've read, that duplicating data between several microservices considered a good idea, because it makes each service more independent.

However, I can't figure out what to do in the following case to provide consistency:

  1. I have a Customer service which has a RegisterCustomer method.
  2. When I register a customer, I want to send a message via RabbitMQ, so other services can pick up this information and store in its DB.

My code looks something like this: 

             ...
            _dbContext.Add(customer);
            CustomerRegistered e = Mapper.Map<CustomerRegistered>(customer);
            await _messagePublisher.PublishMessageAsync(e.MessageType, e, "");
            //!!app crashes
            _dbContext.SaveChanges();
            ...

So I would like to know, how can I handle such case, when application sends the message, but is unable to save data itself? Of course, I could swap DbContextSave and PublishMessage methods, but trouble is still there. Is there something wrong with my data storing approach?

e.e
  • 48
  • 4

3 Answers3

0

Yes. You are doing dual persistence - persistence in DB and durable queue. If one succeeds and other fails, you'd always be in trouble. There are a few ways to handle this:

  1. Persist in DB and then do Change Data Capture (CDC) such that the data from the DB Write Ahead Log (WAL) is used to create a materialized view in the second service DB using real time streaming

  2. Persist in a durable queue and a cache. Using real time streaming persist the data in both the services. Read data from cache if the data is available in cache, otherwise read from DB. This will allow read after write. Even if write to cache fails in worst case, within seconds the data will be in DB through streaming

Saptarshi Basu
  • 8,640
  • 4
  • 39
  • 58
0

NServiceBus does support durable distributed transaction in many scenarios vs. RMQ.Maybe you can look into using that feature to ensure that both the contexts are saved or rolled back together in case of failures if you can use NServiceBus instead of RMQ.

Deb
  • 737
  • 1
  • 5
  • 20
0

I think the solution you're looking for is outbox pattern, there is an event related database table in the same database as your business data, this allows them to be committed in the same database transaction, and then a background worker loop push the event to mq

bz5314520
  • 1
  • 1
  • 2