1

I have this situation:

  • microservice MS1 has a db, let's say with N records, and an external source provides new data that, if valids, should be persisted

  • validation process is performed by a microservice MS2 that is a consumer of a kafka topic T1, on which MS1 sends the new data

  • the validation of the potential N+1 record involves query on all the previous N records. If the validation is successful MS2 produces the result on topic T2, on which MS1 is consumer,so it can persist the new,valid,data.

The problem is the following.

Imagine that the N+1 valid new data is too big and needs a lot of time to be writed on db: it can happen that validation of potential N+2 record fails because querying the db there are only N available record, instead of N+1.

Is it possible, using kafka capabilities in some way, to pause the validation process until MS1 has committed the previous valid data on the db?

Not using kafka, the only way that i have found to ensure validation queries on a up-to-date db is to make rest call between the two microservices, so to wait the each other responses.

Any help or new solution would be really appreciated.

Thanks!

alemoretto
  • 11
  • 1
  • Why is MS2 validating the data for MS1? This sounds a littte strange. Each micro-service should validate its own data which it persiats. Validation is part of the business logic of particular micro-service and should not be spread to 2 services unless you have a very good reason? – xargs Oct 30 '19 at 21:20
  • @xargs because the validation logic in MS2 is shared is used by many other services – alemoretto Oct 31 '19 at 23:04
  • Why is that the case? Usually the vlaidation logic should be sitting together with the rest of the logic of a particular micro-services. If you have some common logic which is validation some common things and used in multiple micro-services you can extract this to a library and reuse it in the micro-services where you need it. Can you tell me what is the criteria how you split your system to micro-services? – xargs Nov 03 '19 at 12:11
  • If I make changes to my validator and it is included as library i have to rebuild and redeploy every application that use it – alemoretto Nov 04 '19 at 22:35
  • I posted an answer with possible options based on your case. – xargs Nov 10 '19 at 11:23

1 Answers1

0

Based on your information that you have provided here I think you have 3 options what you could do here:

  1. Duplicate the MS2 validation logic to MS1

  2. Extract the MS2 validation logic to shared library

  3. Call MS2 micro-service from MS1 directly with a Rest call

Explanation of the options:

  1. Validation logic is one of the essential parts of the business logic and should always be part of the same micro-service which Entities/Aggregates are validated. At first duplication sounds strange and maybe even wrong but in this case it saves you a lot of overhead compared to having it in 2 micro-services. Doing this you would only need to redeploy your MS1(to answer your comment from above). Keep in mind that otherwise network communication between the 2 micro-services will bring you a lot of problems like: performance, delays, network failures, distributed transactions and others.

  2. Extracting the common shared validation logic to some sort of library(for example in .NET nuget, in Node.js npm...) and including it as package in your micro-service MS1 is also an option. This way you will not duplicate the code in all micro-services but if your business requirement for some validation changes you will need to have the change for all micro-services. If that happens you will need to rework your library and/or extract some of the code back to the originator micro-service.

  3. Calling MS2 from MS1 directly using REST API can be done over a Command or Query approach(please read this answer for the explanation on ways of communication between micro-services). There is nothing wrong in calling one micro-service from other over REST and in some cases you have no other choice. Still keep in mind that you will have some delays as you call another api to do your validation.

Conclusion

I would strongly recommend you to use the 1. option and duplicate your validation logic from MS2 to MS1. Validation is part of the Domain logic and should be in the same micro-service for which you have the logic. It can happen that this or similar validation logic is applicable for other micro-services but each micro-service should have its own implementation. The option 2. is mostly used for some business agnostic things like common test infrastructure, common database access(repository classes), common communication mechanism and so on. The option 3. is better if you want to call other micro-service to get some information or to send some command like(Reserve this Product for me or similar), but for doing validation I would avoid that. There is absolutely need for an extra network call just to do validation.

xargs
  • 2,741
  • 2
  • 21
  • 33