Server design suggestions to support multiple workflows managed by multiple teams

Question

Looking for inputs to a design problem. We are redesigning our existing server which has served well so far but won't scale in the future.

Current design: It is one server(we run multiple instances of the same server) which has many workflows. Lets call these workflows A,B,C,D handled inside the server. Until now, we have one development team working on this server which made handling releases easy. Performance is also decent because we leverage in memory caching.

Future design: We now have multiple teams(each team handling one workflow, Team A handling workflow A, Team B handling workflow B and so on). With this new team structure and current design, we are unable to scale our releases(Since its one server, we have only one team releasing at any given time thus reducing overall team efficiency). There is a need for isolation so teams can release their changes to workflows independent of each other. Also, we expect more workflows to be on-boarded into this server.

Any design ideas on how we can solve this problem of ever increasing workflows

My current solution: Split the server into 4 servers so each team can manage the workflows individually. The disadvantage with this approach is the code management. Most of these workflow share common code base. Also, splitting the server causes us to lose out on cache(which is not an issue with current design)

Look forward to hearing your suggestions.

Answer 1

Splitting into different workflow according to different teams makes complete sense. Some advantages that come to mind are:

1. Independent releases like you mentioned.
2. Crashes/Memory leaks/resource hogging from workflow A won't affect workflow B
3. Each Workflow server can be scaled independently. A popular workflow A could be scaled to say more servers while a rarely used workflow B could be working with just 1 server.

There could be more, just pointing out the obvious ones supporting the split.

How to handle disadvantages - let us try to understand with the example of Library Management System. Let us say we need workflows for member borrowing a book, member returning a book, registering a new member.

Most of these workflow share common code base

To resolve this we identify the core common part, in my example I will take definition of book(id, name, field), member(id, name, email). Besides the definitions, I can also have common functions that work on them, like serialisers, parsers, validators.

Now my workflow/s will depend on this common repo. The borrow book workflow will completely be different from add a member workflow, but they will use the same building blocks.

the server causes us to lose out on cache

Exactly what needs to cached and what is the behaviour of the cache is very important.

A fairly static cache(say member cache) can be setup on distributed cache like redis. Say there is a workflow which will identify the close deadlines for the borrowed books and send reminder emails to those members. Once the member ids are identified, their emails could be looked up in the redis cache.

A workflow can have a personalised cache as well. For example during searching for books in the library with the name, the result can be cached in the workflow server only in memory with a TTL, and can be served if same query is asked in near future.

To conclude, the disadvantages you have are nothing but design challenges. I hope that with this random example I was able to give you a few points to wonder upon. Depending on your actual use case, my answer might completely be irrelevant. If so, sincere apologies. :)