4

We have a requirement of building stateless micro services which rely on a database cluster to persist data.

What is the approach that is recommended for redundant stateless micro services(for high availability and scalability) using the database cluster. For example: Running multiple copies of version 1.0 Payment service.

Should all the redundant micro services use a common shared DB schema or they should have their own schema? In case of independent DB schema inconsistency among the redundant services may exist.

Also how can the schema upgrade handled in case of common DB schema?

TechEnthusiast
  • 1,795
  • 2
  • 17
  • 32

1 Answers1

8

This is a super broad topic, and rather hard to answer in general terms.

However...

A key requirement for a micro service architecture is that each service should be independent from the others. You should be able to deploy, modify, improve, scale your micro service independently from the others.

This means you do not want to share anything other than API definitions. You certainly don't want to share a schema; each service should be able to define its own schema, release new versions, change data types etc. without having to check with the other services. That's almost impossible with a shared schema.

You may not want to share a physical server. Sharing a server means you cannot make independent promises on scalability and up-time; a big part of the micro service approach means that the team that builds it is also responsible for running it. You really want to avoid the "well, it worked in dev, so if it doesn't scale on production, it's the operations team's problem" attitude. Databases - especially clustered, redundant databases - can be expensive, so you might compromise on this if you really need this.

As most microservice solutions use containerization and cloud hosting, it's quite unlikely that you'd have the "one database server to rule them all" sitting around. You may find it much better to have each micro service run its own persistence service, rather than sharing.

The common approach to dealing with inconsistencies is to accept them - but to use CQRS to distribute data between microservices, and make sure the micro services deal with their internal consistency requirements.

This also deals with the "should I upgrade my database when I release a new version?" question. If your observers understand the version for each message, they can make decisions on how to store them. For instance, if version 1.0 uses a different set of attributes to version 1.1, the listener can do the mapping.

In the comments, you ask about consistency. This is a super complex topic - especially in micro service architectures.

If you have, for instance, a "customers" service and an "orders" service, you must make sure that all orders have a valid customer. In a monolithic application, with a single database, and exclusively synchronous interactions, that's easy to enforce at the database level.

In a micro service architecture, where you might have lots of data stores, with no dependencies on each other, and a combination of synchronous and asynchronous calls, it's really hard. This is an inevitable side effect of reducing dependencies between micro services.

The most common approach is "eventual consistency". This typically requires a slightly different application design. For instance, on the "orders" screen, you would invoke first the client microservice (to get client data), and then the "orders" service (to get order details), rather than have a single (large) service call to retrieve everything.

Neville Kuyt
  • 29,247
  • 1
  • 37
  • 52
  • Many Thanks for the detailed answer. Just to quote you "each service should be able to define its own schema". If there are redundant MS of "same version" running is it an issue? e.g., Multiple instances of "Product" service of version 1.0 using a shared schema. I understand that upgrade to a different version would be an issue which possibly can be handled using rolling upgrade techniques. – TechEnthusiast Sep 07 '17 at 14:53
  • *( Observation )* If one can safely remain to "just-read" the evangelisation, without a must to live according to that rules in The Real World ( business domain ), the story is not much different from **"The Day of Triffids"**. Yes, disintegrated responsibility, fragmented into isolated groups, if not opportunistic soloists. **If NASA were using this approach during Man to the Moon-programme**, the famous sentence would sound a lot different ... "Это был небольшой шаг для человека, но гигантский скачок для человечества" or "这对一个男人来说是一小步，但是对人类而言是一个巨大的飞跃" There is no space for handling remedies – user3666197 Sep 07 '17 at 14:55
  • Extending a responsibility for a service design also to a service-operations ( where designer has no powers outside of atomicised service-scope ) is frustrating managerial absurdity, like making blinded-Triffid-victims responsible for reading something, they will never see on their own ( principally limited scope of visibility ( yes, blindness ) ). **'ve seen many great Projects' frustrated once having fallen into a next-set-of-beliefs derived from similarly *( just )* evangelisated *truths*** ( Sharing pieces of experience may help reduce pains from "just" repeating the same errors ) – user3666197 Sep 07 '17 at 15:02
  • 1
    In general, you want to maintain the fewest possible sources of data - so I would recommend upgrading the schema if necessary, rather than keeping many versions of the same schema running. I've updated the answer. – Neville Kuyt Sep 07 '17 at 15:10
  • @NevilleKuyt Great answer. Can you please help with the scenario where the MS instances use a highly available cloud database as a service(DBaaS) which provides necessary consistency level. Would it still be a bad idea to use shared schema for a particular version? In our case consistency for API consumers is very critical. – TechEnthusiast Sep 07 '17 at 15:18
  • Can you be more specific about "consistency"? Perhaps give an example? – Neville Kuyt Sep 07 '17 at 15:40
  • @NevilleKuyt For us consistency means that if different clients make 3 simultaneous calls to the redundant MS then they should get consistent results. i.e., Strong consistency at service (provided by multiple redundant microservices) level. – TechEnthusiast Sep 08 '17 at 01:34
  • @NevilleKuyt Many thanks for the answer. In case we want the client code not to handle the consistency aspect then would it be a bad idea going for shared schema for the REDUNDANT microservices(e.g., copies of "Payment" microservice). Do people follow this approach even though it is not a recommended approach in a microservices based system. – TechEnthusiast Sep 08 '17 at 10:10
  • 1
    To be clear - multiple instances of a service (e.g. Payment) can access a single data store, using the same schema. That keeps the dependency within that logical microservice. But having multiple microservices (e.g. payment and orders) accessing the same data store, and the same schema, is a bad idea - you're creating coupling at the lowest possible level, and the whole point of micro services is to make them independent. – Neville Kuyt Sep 08 '17 at 10:45