0

I have two (micro)services A and B where service B is interested in data of service A. Let's say service A has a table of products with a lot of fields. Service B wants to store a table of products with a subset of the fields (name and id).

To keep service B up to date, my intention is that service A will send messages to the message bus for create / update / delete of products. This seems to be a common approach.

But now there is the problem of an initial sync (or maybe once a resync), because service B is a new service and service A already exists. I do not find any best practice on that matter. I can imagine two solutions:

Solution 1: Service B pulls the initial data via the additional REST API which service A also offers. Disadvantages i see, is that i need to configure the URL and as well service A has to be running at the moment, so i would not prefer this solution.

Solution 2: Service B sends a message with the intention "SyncProductRequest". Then service A sends for each product an update or sends all product at once in one message. Sending all at once probably is not a good idea because of the message size. On the other hand that way i could easily remove products which are not existent anymore in service A. As well i see the problem that it could occur a situation where one product is changed at the same time as sync is happening. This problem i think i could only solve by having a modified date time on the product.

Would you solve this "initial sync problem" with solution 2 as i described, or do you know kind of a best practice for this type of microservice problem?

In case it matters, i am using; .net and masstransit with rabbitmq

Diego Frehner
  • 2,396
  • 1
  • 28
  • 35
  • Perhaps consider a more events based approach rather than a command driven approach? So rather than sending commands such as "create/update/delete" send an event such as product updated/removed. You haven't mentioned why you need an initial sync? On receiving a product updated event, if it doesn't exist in service B then create it. If it does then update it, etc. – robs Apr 27 '22 at 05:09
  • Hey rob, for create/update/delete I implemented an event driven approach as you suggest. As well i handle updates as update or create as you suggest. And it you're right it can be sufficient, (in my first use case which was a new feature and i received all relevant message, because service A + B were changed and deployed at the same time. My use case where i need an initial sync is, service B now suddenly needs to know partly about user / organisation structure. These information is existing and managed by service A. – Diego Frehner Apr 28 '22 at 18:43
  • I think the problem generally occurs when multiple services require partly the same models and the whole system is evolving over time, not written at once. – Diego Frehner Apr 28 '22 at 18:44
  • As well a resync could be required once you had a mistake in your event logic and you need to fix that. – Diego Frehner Apr 28 '22 at 18:49
  • Yes, there can/will be scenarios where services have same/similar models. If you share a model from the same database or schema across services you will loose some benefit of loosely coupled services and increase dependencies between services. The decision here is yours to make. If you are seeking a micro-service approach for those benefits then start from loose coupling and only break that if you really feel you have no other way. Otherwise keep loose coupling and provide ways to update or sync the service models. You may find your shared models are only partially shared, reducing the sync. – robs May 01 '22 at 06:40
  • A shared db/schema/table will require you to update all services that use it if the model changes. However, it is not a totally invalid approach. You can take steps to mitigate the dependency by only allowing one service to write data, others are read only & reduce the interface footprint, perhaps by defining a view in the db that is sufficient for read only services. Another alternative is to have an org structure service that handles request/response queries for org data from other services. – robs May 01 '22 at 06:50
  • For event correction, arguably the most important is to design your events to be idempotent and use a message broker that guarantees at least once delivery. RabbitMq & Azure Service Bus are two examples. If you are not using a message broker you will need to implement message acknowledgment in the services yourself. A message broker will go along way to solving/removing event logic errors for you. – robs May 01 '22 at 06:55

1 Answers1

0

What makes sense is the way you are describing by sending some SyncProductRequest. Your service A can listen to that message and start dispatching Inserted/Deleted/ProductEvent (same event as for "normal" events). You might add some additional property that this is a synchronisation response so that other microservices don't create unnecessary load and can filter those messages.

I would stick only to event messaging (rather adding rest) because you can keep your implementation for the same purpose of consuming insert events.

Be aware that there might be race conditions between insert/deletes depending on your message infrastructure and ordering guarantees.

emilio
  • 588
  • 3
  • 12
  • thats how I do it meanwhile. I like the idea of adding an additional property to indicate that the message is a synchronisation response. – Diego Frehner Jan 06 '23 at 19:07