0

Little context - today I have a monolith application that I am planning to split into micro-services due to it's growth and the need to partial re-deployments.

I'm designing a development process where I have a number micro-services in a multi repo environment (All written in python).

  • There is a one "Foundation" repository that stores ~30 different packages of shared code.
  • Each micro-service is stored in a different repo.
  • The plan to share the code is using a package manager ("private" PyPi) where each package manages it's semantic version and on change it's packed and published.

An example of the packages and dependencies:

"Foundation":
Package 1
Package 2 --> 1
Package 3 --> 1,2
Package 4

"Micro-service 1"
Package 5
Package 6 --> 5,1,3

"Micro-service 2"
Package 7 --> 4

I'm trying to understand how to handle the flow where Package 1 introduces new change (let's say - feature change that causes a minor version bump) How I should keep up with changing all the depended libraries ? considering:

  1. Package 3 depends on 1 directly and indirectly - think i want to avoid case where they have different versions - it will cause a problem in the deployment.
  2. Updating all the relevant micro-services, for example Package 6 depends on 1.
  3. Every dependent package may introduce different version bump, some of them may have only patch level change due to Package 1 minor change.

It seems that in a fresh development process where there are a lot of changes it will be a nightmare to keep up updating everything I've seen some different scripts or tools like Lerna for npm that should help with that - but still don't understand what's the best strategy for fresh development projects

sborpo
  • 928
  • 7
  • 15
  • What is the nature of the interface(s) between these micro-services? – jwdonahue Jul 04 '20 at 14:45
  • they interconnect via Events and direct APIs for commands - for example "project added", "project finished" etc. The shared libraries are usully used for Message Queues wrappers, logging handlers, different sanitizers, parsers ,etc – sborpo Jul 04 '20 at 18:31
  • Are the interfaces changing or just the behaviors? – jwdonahue Jul 04 '20 at 21:07
  • both ways, sometimes we may break the "major" part of the semantic version – sborpo Jul 07 '20 at 04:13
  • 1
    This is just too broad a problem for there to be a good SO type of answer for. There's no end to the degrees for freedom here when trying to find a solution. Much depends on your tool-chain capabilities and how committed you are to using those tools. The [diamond dependency problem](https://en.wikipedia.org/wiki/Multiple_inheritance#The_diamond_problem) is provably hard at scale. The fact you're pushing it out to the package layer doesn't help much. This topic covers the entirety of DevOps and is therefore worthy of a library full of books. – jwdonahue Jul 07 '20 at 18:06
  • It will be easy for you to contact me directly. Send an email with "sborpo" in the subject line and I'll respond with some things you need to be thinking about. I've tried to help folks here with similarly broad/complex problems and the threads always wind up getting closed or deleted. I am not available for any paid consulting jobs at the moment, and I especially don't ever want to do another deep dive into anything Python related, so I promise you wont be hit with a sales pitch. We'll play a game of <20 questions and then maybe I can offer some tips. – jwdonahue Jul 08 '20 at 01:00
  • I have sent a message on your info mailbox, Thanks a lot! – sborpo Jul 10 '20 at 21:06

0 Answers0