1

I am trying to set up a complex build pipeline, the dependency graph is shown as below.

      Prj-A
        |
      Prj-B
  /     |    \ 
Prj-C Prj-D Prj-E
 |      |     |
 |      |     |
 |      |     |
 --------------
        |
      Prj-F

The pipeline starts with Prj-A, followed by Prj-B. Prj-C D E can be run in parallel. When Prj-C, D, E are all completed, run Prj-F.

One of the requirements is, for example, since Prj-C depends on Prj-A and Prj-B, i'd like to only run Prj-C then Prj-G if only the source code in Prj-C is changed. This is to avoid building other jobs to save the whole build time.

After a lot of googleing, I didnt find any solution is available for such cases. So my solution in mind (I don't try it) is:

  1. Create a trigger job for each project, which only check if the source code for that project is changed, so we will have 7 trigger jobs.
  2. Create a dependency resolver job for the pipeline.
  3. When any of the trigger job detects a change, it triggers the dependency resolver job. The dependency resolver job will know which project is changed, and determines the projects can be skipped in this build, and set corresponding environment variable. For example, when Prj-C is changed, it sets skip-proja=true, skip-projb=true, skip-projd=true, skip proj-e=true. After that, it triggers the build pipeline.
  4. When each project is run, it checks it can be skipped according to the environment variable set by the dependency resolver job.

Hope I have made myself understood, please let me know if there are other solutions for this case or your comments on my solution. Thanks!

Mingjiang Shi
  • 7,415
  • 2
  • 26
  • 31

1 Answers1

1

Your solution seems workable but ideally you want proj-f to trigger with only that combination of C, D and E that have all passed from the same version (build counter) of B.

For such complex setups, it is better to use a tool like Go (I work there) that is built ground up with these scenarios in mind. You have a fan-in scenario that can give rise to spurious builds unless you have a tool that understands fan-in.

http://support.thoughtworks.com/entries/22229668-Go-s-Dependency-Management

http://www.thoughtworks-studios.com/go-continuous-delivery/compare

ottodidakt
  • 3,631
  • 4
  • 28
  • 34