0

I have an architecture based on several Self-Contained Systems (Microservices, if you want), each one of them owning a given set of data.

We would like to implement a reporting database (with near-realtime updates) by integrating all of the systems to let them "project" their data into the centralized reporting data storage.

I would like to give to each system the responsibility to fill their piece of the reporting data model, in a loosely coupled manner. Indeed, I would like to avoid implementing a centralized "reporting system" which maybe can listen to MQ messages coming from the various systems and fetch data trough API from them (acting as an orchestrator basically).

I would like to do the opposite, meaning... the various systems should have the possibility to plug their "data reporting projector" component into this reporting system.

This way I could avoid dependencies between the various systems that want to project data and the reporting system itself (and vice-versa) and keep the logic for the reporting projection in the owning system itself.

Do you know any already existing tools I could use in order to implement this kind of integration layer for reporting purposes?

Can you suggest some architectural patterns or approaches that can be followed in this case?

The main technology we use for the backend is .NET.

Thanks.

Stefano
  • 83
  • 1
  • 5

1 Answers1

0

A very broad question, but I will attempt an answer.

We have a system which exposes a connector template git repo.

Inside the connector template code, there are event handlers which fire whenever some "interesting things" happen inside the system.

Every system which is required to know about these "events" will copy the template, implement their own handling code inside the handlers, and then commit the code back into our source control after we have reviewed the pull request. These implementations are then packaged and deployed with our system.

When an interesting thing happens, the system then finds all implementations of the connector template and invokes the relevant handler for that event against each one in turn.

The handling code is bespoke to each integration partner, and most of them decide to handle these events by calling http endpoints in their own systems.

For your use case, you could have a similar approach, but instead of defining different "things which happen" in your reporting system, the only event in your connector template is a "give me the latest data" event, which fires on a schedule.

Each integration partner - in your use case, the teams responsible for your source systems - would then implement their own handler, which would probably call out to the source system and get the data needed by the reporting system.

The benefits of this approach:

  • Implementation of integration mechanism rests with the source system teams
  • Ownership of DTO types rests with the reporting system team
  • Changes to integration code will always go through the reporting system team
tom redfern
  • 30,562
  • 14
  • 91
  • 126