microservices consuming different version of same service events

Question

I have following scenario

1. Account Service handles user accounts
2. Reports Service for handling historical reports

When new user is registered via Account Service it triggers a Kafka event which is consumed by Reports Service which inserts one record in a table in reports own database.

Now we are introducing versioning (maximum 2 version , 1.0 and 2.0) for zero down time deployments.

• Account Service 1.0 produces event E1
• Account Service 2.0 produces event E2

On the consumer side,

• Reports Service 1.0 consumes E1

Lets say Reports Service has decided it does not want to bump the version because it has no changes for version 2 hence there is only Reports Service 1.0 running. Both report versions (if deployed) use the same database.

Given above scenario, when a new user registers via Account service 2.0's this new user cannot be consumed and ingested into Reports Service database as E2 is lost not consumed by reports service.

How do we handle this scenario?

Answer 1

There are a few approaches you can take:

• You can implement a service which serves as an anti-corruption layer (corruption in this sense being more in the direction of moral purity than data quality: the corruption that it's preventing is the report service having to care about other services changing the schema) between the account service and the report service. This service previously would translate E1 into C, the type of command which the report service consumes (this translation could be the identity transformation: C is in fact E1). Then, since E1s are no longer being emitted by the account service, you could replace this anti-corruption layer with a translator from E2 to C. Since the anti-corruption layer is in principle stateless, it's a much smaller lift to change this than to change the report service.

• Alternatively, and this is more of an option if this is a hypothetical or you're gaming out how to handle a schema change, you can require that account service publish both E1 and E2 events for the benefit of consumers with a preference. This could be implemented by, in CQRS terms, having a read-model which is the E2 events translated into E1s.

• It may be useful to consider serializing events using a format that allows for an E2 to be deserialized into an E1. For instance, a convention of only adding fields to JSON schemas and having JSON readers ignore unexpected fields can work well.

• As a last resort, if none of those is viable, then in actual fact you've painted yourself into the corner of having the account service and the report service be pretty tightly coupled: a change in one (with fairly high certainty) ripples into the other. You may want to pursue a new architecture along the lines of the first two options, or, alternatively, put a lot of extra effort into getting the problem solved right the first time so you don't have to do new events. Your situation is partially summed up by Mathers, 2002:

You only get one shot... the opportunity comes once in a lifetime.

The first two approaches (which are what I'd tend to recommend) are basically the same: the difference is organizational responsibility for maintenance.