3

In our organization, we maintain one git repository from which we build multiple microservices. We use Java (spring boot) and maven for building the services.

Each microservice resides in its own maven module + there are "common" (infrastructure) modules that appear (included as dependencies) in all our microservices.

Currently, we don't promote versions of maven artifacts: always run with 0.1-SNAPSHOT in maven, and have a CD process, i.e. each commit reaches our servers

Now, I'm looking for effective way to build our stuff.

Currently, we have Jenkins with slaves that are running on demand in the cloud upon each commit. We have one Jenkins job per microservice.

When someone executes a commit we trigger builds for all microservices, even if the committed files are used only by one microservice with the following command: 

mvn --projects backend/microsevice-a --also-make
mvn --projects backend/microsevice-b --also-make
....
mvn --projects backend/microsevice-N --also-make

So, we build the microservice with all its dependencies every time, which is not optimal.

I'm looking for the way to use build only the modules that contain the code that has been changed during the last commit (s)

One solution we thought about is using nexus/artifactory and introduce the full-fledged support for versions, so that every time jenkins builds its stuff it will publish the version in, say, artifactory so that later jenkins will be able to take the "ready" versions out of artifactory, but this looks to be a very complicated approach, because I won't be really able to track what dependency in what revision has been deployed with my microservice.

Since I'm not a DevOps/build guy, I'm more towards the following approach:

  • Build some plugin (maven/Jenkins) - that can parse the information out of last commit, analyze the dependency tree of all the backend project and figure out which microservices should be re-built, but It looks like custom development, I'm not sure what are other people do, maybe its already available in Jenkins/maven - I don't know...

Please share your thoughts about these approaches/propose alternatives A help is appreciated :)

Mark Bramnik
  • 39,963
  • 4
  • 57
  • 97
  • 1
    Use separate repositories or separate branches. Alternatively you could write script that would check if last commit was made on files below some directory structore as precondition for building. – Antoniossss Jun 12 '18 at 14:12
  • Precondition can be done using git https://stackoverflow.com/a/16343950/1527544 – Antoniossss Jun 12 '18 at 14:15
  • Well, if you are running a monorepo, it might be worth to checkout https://bazel.build/ which can cache build results. We have one and it goes pretty good. Besides, with microservices, build code once, test container images with every new version – Urosh T. Jun 12 '18 at 14:20
  • I also suggest go by separate repo/Micro-service,which will give great flexibility with versioning and easier pipeline code to maintain. – Naveen Kumar Katragadda Jun 12 '18 at 15:16
  • Thanks for answering, For now, our organization uses "mono" repository, and while multi-repo can be a feasible approach in a long-term, currently we're not there. Leaving aside a debate of mono vs. multi repo I'm primarily interested in mono repository approach. – Mark Bramnik Jun 13 '18 at 03:05
  • Regarding the last commit checks by git script, yes its can be done, but if some file in some module in infra changes , like for example "infra/http/extensions/SomeFile.java", how do I know that only certain microservices, like microservice-a, microservice-f, microservice-x, should be rebuilt (only because they're dependent on infra-http-extensions module) , while other microservices who don't use http at all should not be recompiled? I think this information can only be obtained by analyzing maven dependencies, hence I thought about the maven plugin implementation... – Mark Bramnik Jun 13 '18 at 03:10
  • https://github.com/lesfurets/partial-build-plugin lets you do stuff like building only modules affected by last commit and their dependencies or building all modules, but running tests only on affected modules. – arseniyandru Jul 15 '19 at 21:40
  • @MarkBramnik How did you solve the problem? – Ghasem Sadeghi Oct 11 '20 at 19:33

0 Answers0