Building dependent projects using Jenkins

Question

Lets say i have 3 ANT projects each with its build.xml script located at the top level folder

proj1
 file1.java
proj2
 file2.java
proj3
 file3.java

I configured Jenkins to build these 3 projects and it builds them and stores the output in three separate folders.

/output/proj1/2012-02-28_20-11-06/proj1/distribution/proj1.jar
/output/proj2/2012-02-28_20-11-08/proj2/distribution/proj2.jar
/output/proj3/2012-02-28_20-11-09/proj3/distribution/proj3.jar

The ANT script is the one that puts the jar file in the distribution folder. I want to use these jar files in another project which produces a war file.

The jar files that are generated in each project are used to build a WAR file. Is there any way i can configure Jenkins to build proj1, proj2, proj3 then copy the jar files to a specific location which will contain another ANT project which when run will produce a war file? Or does this have to be coded with ANT?

Also, is it possible to configure jenkins to do multiple checkouts before running the ANT build script. I am using CVS and would like to do 3 checkouts each using a different tag before running the build.

Edit

I posted the question last night before i went to bed. Reading it again today i think it is not clear enough so i will clarify the requirement again. I have 4 projects that use ANT as the build mechanism and are all stored in a CVS repository. All 4 projects when built form a WAR file that is deployed on a Tomcat application server.

proj1
 - file1.java
 - file2.java

proj2
 - file3.java
 - file4.java

proj3
 - file3.java
 - file4.java

proj4
  - jsp
    - home.jsp
 - css
    - home.css
  - WEB-INF
     - lib
     - classes 
    - web.xml

Each project (except proj4) has an ANT build file which builds the project and produces a JAR file which it stores in a 'distribution' folder on the root folder of the project. For example after proj1 is built it will look like this

proj1
 - file1.java
 - file2.java
 - distribution
      - classes
           - file1.class
           - file2.class
      - proj1.jar

Proj4 has a dependency on proj1, proj2 and proj3 in that it first needs to copy all the jar files into its workarea before it builds a war file. The ANT build file for project4 copies the JAR files from proj1, 2 and 3 and builds a WAR files. The resulting WAR file contains the following structure

proj4
  - jsp
    - home.jsp
 - css
    - home.css
  - WEB-INF
    - lib
       - proj1.jar
       - proj2.jar
       - proj3.jar
    - classes 
    - web.xml

Now to automate the above i need to somehow configure Jenkins and ANT to do the following (in the order shown)

- Checkout proj1 from CVS using RELEASE1.1 tag
- Build proj1

- Checkout proj2 from CVS using RELEASE1.2 tag
- Build proj2

- Checkout proj3 from CVS using RELEASE1.1 tag
- Build proj3

- Checkout proj4 from CVS using RELEASE1.1 tag
- Copy the proj1.jar, proj2.jar, proj3.jar from the previous 3 builds and copy them to the proj4 WEB-INF/lib folder.
- Build proj4
- Deploy the WAR file.

I played around with Jenkins yesterday and managed to configure it to build each of the projects but each jar file ends up in an unrelated folder. How can i configure Jenkins to build the projects as i've described above?

The environment has the following properties

Build tool - ANT
OS - Solaris

Maven would probably work with this but at the moment there are no plans to introduce Maven. If there is a simple Maven like build tool that i can use to just store the libraries i need that does not require external access then please let me know.

Thanks in advance.

malenkiy_scot · Accepted Answer · 2012-02-28T22:35:19.123

4

I do not understand exactly what your requirements are, but let's give it a shot:

Start your three jobs from a single job (lets call it TriggerJob) using Parameterized Trigger Plugin as a build step. Give them TriggerJob's BUILD_ID as a parameter, which they'll use to store the results (instead of using their own BUILD_ID's; if you still need to build them separately sometimes - without being triggered from TriggerJob - you can specify some default value for the parameter, and check for it with Conditional BuildStep Plugin, when it has the default value - use own BUILD_ID, otherwise use the parameter passed in).

Make TriggerJob wait for the three jobs to finish, then collect your jar's (TriggerJob knows where they are), and proceed with the fourth job that will call your last ANT to archive them together (or make it into an ANT build step in TriggerJob, whatever fits better).

Just in case the following observation may prove useful for you: every ANT build step can use its own build file (it's under ADVANCED button). You can use environment variables to specify its path.

Multiple checkouts: I'm not familiar with CVS plugin, but with Subversion plugin I prefer to do things by hand: I have trigger jobs that poll SVN via Subversion Plugin and pass revision numbers as parameters to jobs that do actual work. Those jobs then use explicit SVN calls (from shell build steps) to update to revisions passed as parameters. Maybe you can do the same thing with CVS plugin.

edited Feb 28 '12 at 22:35

answered Feb 28 '12 at 22:29

malenkiy_scot

16,415
6
64
87

I have clarified the requirement a bit more so hopefully it is clear now. See the Edit. How exactly will the TriggerJob know where the jar files are? – ziggy Feb 29 '12 at 10:17
_but each jar file ends up in an unrelated folder_ - what do you mean? It ends up in some place other than what your specify on top (/output/... etc.)? Which place? – malenkiy_scot Feb 29 '12 at 11:05
When jenkins runs every project, it checks out the files from CVS and puts them in a workspace folder for that project. When the ANT script is run the JAR file ends up in the workspace folder. For example proj1's jar file will be in workspace/proj1/distribution/proj1.jar and proj2's jar file will be in workspace/proj2/distribution/proj2.jar - Proj4 can not build the war file with the jar files in these folder. (unless i manually copy them to the proj4 workspace) – ziggy Feb 29 '12 at 11:18
That's what I thought. There are two options, as I see it: run your projects in custom workspaces (in job configuration, under *Advanced Project Options* push _Advanced_ button and check _Use custom workspace_), then enter a path for a custom workspace. I do not like this option since theoretically workspace is considered temporary and it is not a good practice to rely on its persistence. – malenkiy_scot Feb 29 '12 at 11:46
The second option: copy your jar's (and other artifacts) in a post-build action to wherever you want with [Copy Artifact Plugin](https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin) – malenkiy_scot Feb 29 '12 at 11:50
Doing it all with Jenkins seems to be overkill. Would you recommend using a build repository like Maven or Ivy instead? – ziggy Feb 29 '12 at 14:27
It's what you are used to and comfortable with. I'm used to Jenkins and doing tricks with it, other people are used to other things. I know nothing about Ivy and have a rather limited experience with Maven, so I can't tell. Those other approaches may fit your working style better. What I can tell you for sure that no matter what you use unless you do some basic vanilla stuff you'll have to get creative, look under the hood, etc. – malenkiy_scot Feb 29 '12 at 15:06

score 2 · Answer 2 · edited May 23 '17 at 09:59

2

Use Apache Ivy with proj1, proj2 and proj3 publishing their jar files to an artifact repository. You can then proj4 triggered, whenever you publish a new version of proj1, proj2 or proj3 (e.g., using the Jenkins Ivy Plugin). Proj4 again uses Ivy, but this time to retrieve the proj1, proj2 and proj3 jars as dependencies, which are then used to build the WAR (which you would also publish to your repository).

If you are feeling confident, you can also consider switching from Ant to Gradle, which also uses Ivy under the covers. You might also consider Maven, but I'm not familiar enough with it to recommend it. I've heard a lot of people swear by it and a lot of people swear at it.

Here is a similar question and answer

edited May 23 '17 at 09:59

Community

1
1

answered Feb 29 '12 at 10:35

Tom Howard

6,516
35
58

I did consider Maven but decided against it because of the requirement for it to have external access or having to maintain a local repository. Does Ivy work the same way or can i configure it to just use the libraries i have locally? If Ivy can be configured to only work with libraries i provide it then i could use it. – ziggy Feb 29 '12 at 11:06
1

You're publishing jars to a common distribution area. Ivy could be configured to use this as a local repository. It's tricky but worth the effort. Finally ivy also has a buildlist task that could automatically determine your project's build order by checking interdependencies declared in your module's ivy files. – Mark O'Connor Feb 29 '12 at 20:15
@MarkO'Connor is correct. If you don't want to maintain a repository, then you can tell it to use the local repository. When publishing and retrieving from the local repository, Ivy just stores the jars in ${user.home}/.ivy2/local (though it can be configured to be elsewhere). – Tom Howard Feb 29 '12 at 20:43

Building dependent projects using Jenkins

Edit

2 Answers2