maven dependencies copied from dependent project after zip assemby

Question

If this already has an answer, I haven't managed to find it. I have spent many hours getting this far, before throwing in the towel and asking here! When it comes to Maven, I would describe myself as a 'Sunday driver'.

Plugin versions: compiler=3.9.0; resurce and dependencies=3.2.0; jar=3.2.2; assembly=3.3.0.

I have two Maven projects, let's call then AppA and Proj1. Proj1 contains all of the 'working' code and 3rd party jar dependencies.

AppA contains the Main class and the app's folders such as 'conf' and 'logs'. Both projects have 'jar' packaging.

AppA's pom has the plugins required to create the jar file with a manifest that defines all of the required jar files in its classpath as 'lib/xxx.jar'. It also has 'Proj1' as a dependency.

The problem I have is that Maven is assembling the zip file before copying all of the dependent jars to the 'lib' folder. Which means that the 'lib' folder is missing from the zip file.

If I build AppA from a single project, the zip file is assembled after the 'lib' folder has been populated,

Can anyone advise me whatI need to do to persuade Maven to copy the dependent jar files to 'lib' before assembling the zip file?

The reason that I have this structure is so that I can create AppB + Proj1 in the future.

Also, the lib file contains all of the Maven plugin jars and their dependencies. When I buils from a single project, they are excluded.

[pom.xml]

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.w3p.njams</groupId>
<artifactId>com.w3p.iib.njams.client</artifactId>
<version>Beta-1.0.1.0</version>
<packaging>jar</packaging>
<name>nJAMS Client App for IIB</name>
<description>nJAMS Client App for IIB</description>

<properties>
    <jdk.version>1.8</jdk.version>
    <maven.compiler.version>3.9.0</maven.compiler.version>
    <njams.client.version>Beta-1.0.1.0 </njams.client.version>
    <client.build.dir>njamsIIBClient</client.build.dir>
    <ibm.api.artifact>com.w3p.api.iib10</ibm.api.artifact>
    <ibm.api.version>Beta-1.0.1.0</ibm.api.version>
    <dependency.version>3.2.0</dependency.version>
    <resources.plugin.version>3.2.0</resources.plugin.version>
    <lifecycle.mapping.version>1.0.0</lifecycle.mapping.version>
    <!-- ** Logging ** -->
    <log4j.version>2.17.1</log4j.version>
    <disruptor.version>3.4.4</disruptor.version>
</properties>

<dependencies>

    <!-- ** Logging ** -->
    <!-- https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-core -->
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>${log4j.version}</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-slf4j-impl -->
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-slf4j-impl</artifactId>
        <version>${log4j.version}</version>
        <scope>test</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/com.lmax/disruptor -->
    <!-- for async logging -->
    <dependency>
        <groupId>com.lmax</groupId>
        <artifactId>disruptor</artifactId>
        <version>${disruptor.version}</version>
    </dependency>

    <!-- ** The nJAMS Client common to IIB and ACE -->
    <dependency>
        <groupId>com.w3p.njams</groupId>
        <artifactId>com.w3p.njams.client</artifactId>
        <version>${njams.client.version}</version>
    </dependency>
    <!-- ** W3P's IIB/ACE API ** -->
    <dependency>
        <groupId>com.w3p.njams</groupId>
        <artifactId>${ibm.api.artifact}</artifactId>
        <version>${ibm.api.version}</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-dependency-plugin -->
    <dependency>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <version>${dependency.version}</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-resources-plugin -->
    <dependency>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-resources-plugin</artifactId>
        <version>3.2.0</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-assembly-plugin -->
    <dependency>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-assembly-plugin</artifactId>
        <version>3.3.0</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-source-plugin -->
    <dependency>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-source-plugin</artifactId>
        <version>3.2.1</version>
    </dependency>
</dependencies>

<build>
    <!--        <pluginManagement>  -->
    <plugins>
        <!-- *** In Build Sequence *** -->
        <!-- Maven Resources Plugin  - copies resources fron Eclipse project folders to output build folder = cliemt-->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-resources-plugin</artifactId>
            <version>${resources.plugin.version}</version>
            <executions>
                <execution>
                    <id>copy-resources</id>
                    <!-- bind to the validate phase -->
                    <phase>validate</phase>
                    <goals>
                        <goal>copy-resources</goal>
                    </goals>
                    <configuration>
                        <includeEmptyDirs>true</includeEmptyDirs>
                        <outputDirectory>${project.build.directory}/${client.build.dir}_${project.version}</outputDirectory>
                        <nonFilteredFileExtensions>
                            <nonFilteredFileExtension>cache</nonFilteredFileExtension>
                            <!-- serialised FlowTtoProcessModelCache -->
                            <nonFilteredFileExtension>pmd</nonFilteredFileExtension>
                            <!-- serialised ProcessModels -->
                        </nonFilteredFileExtensions>
                        <resources>
                            <resource>
                                <targetPath>${project.build.directory}/${client.build.dir}_${project.version}/conf</targetPath>
                                <directory>conf</directory>
                                <filtering>true</filtering>
                                <exclude>log4j2-test.xml</exclude>
                                <exclude>njams*.xml</exclude>
                            </resource>
                            <resource>
                                <targetPath>${project.build.directory}/${client.build.dir}_${project.version}/flowToProcessModelCache</targetPath>
                                <directory>flowToProcessModelCache</directory>
                                <filtering>true</filtering>
                                <include>dummy.cache</include>
                            </resource>
                            <resource>
                                <targetPath>${project.build.directory}/${client.build.dir}_${project.version}/certs</targetPath>
                                <directory>certs</directory>
                                <filtering>true</filtering>
                                <include>dummy.cert</include>
                                <exclude>*-endpoint</exclude>
                                <exclude>*-instanceId</exclude>
                                <exclude>*.key</exclude>
                                <exclude>*.pem</exclude>
                            </resource>
                            <resource>
                                <targetPath>${project.build.directory}/${client.build.dir}_${project.version}/logs</targetPath>
                                <directory>logs</directory>
                                <filtering>true</filtering>
                                <exclude>njams*.log</exclude>
                            </resource>
                            <resource>
                                <targetPath>${project.build.directory}/${client.build.dir}_${project.version}/images</targetPath>
                                <directory>images</directory>
                                <filtering>false</filtering>
                            </resource>
                            <resource>
                                <targetPath>${project.build.directory}/${client.build.dir}_${project.version}/jms</targetPath>
                                <directory>jms</directory>
                                <filtering>true</filtering>
                                <include>JNDI_Local/*.bindings</include>
                                <include>JNDI_Remote/*.bindings</include>
                            </resource>
                            <resource>
                                <targetPath>${project.build.directory}/${client.build.dir}_${project.version}/monitoringProfiles</targetPath>
                                <directory>monitoringProfiles</directory>
                                <filtering>true</filtering>
                                <include>dummyProfile.xml</include>
                                <include>*.xsd</include>
                            </resource>
                            <resource>
                                <targetPath>${project.build.directory}/${client.build.dir}_${project.version}/processModels</targetPath>
                                <directory>processModels</directory>
                                <filtering>true</filtering>
                                <include>dummy.pmd</include>
                            </resource>
                            <resource>
                                <targetPath>${project.build.directory}/${client.build.dir}_${project.version}/scripts</targetPath>
                                <directory>scripts</directory>
                                <filtering>false</filtering>
                            </resource>
                        </resources>
                    </configuration>
                </execution>
            </executions>
        </plugin>

        <!-- ** Maven Compiler Plugin ** -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>${maven.compiler.version}</version>
            <configuration>
                <source>${jdk.version}</source>
                <target>${jdk.version}</target>
            </configuration>
        </plugin>

        <!-- Maven Dependency Plugin -> copies dependenciea to  'appName'_${project.varsion}  -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
            <version>${dependency.version}</version>
            <executions>
                <execution>
                    <id>copy-dependencies</id>
                    <phase>install</phase>
                    <!-- prepare-package -->
                    <!-- waspacjage -->
                    <goals>
                        <goal>copy-dependencies</goal>
                    </goals>
                    <configuration>
                        <outputDirectory>${project.build.directory}/${client.build.dir}_${project.version}/lib</outputDirectory>
                        <overWriteReleases>false</overWriteReleases>
                        <overWriteSnapshots>false</overWriteSnapshots>
                        <overWriteIfNewer>true</overWriteIfNewer>
                        <!-- The next line actually excludes the scope 'test' jars from the build -->
                        <includeScope>compile</includeScope>
                    </configuration>
                </execution>
            </executions>
        </plugin>

        <!-- Maven Jar Plugin - Create the jar file and it's manifest entries -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-jar-plugin</artifactId>
            <version>3.2.2</version>
            <configuration>
                <outputDirectory>${project.build.directory}/${client.build.dir}_${project.version}</outputDirectory>
                <finalName>${client.build.dir}-${project.version}</finalName>
                <excludes>
                    <!-- -->
                    <exclude>**/*.properties</exclude>
                    <exclude>**/*.xml</exclude>
                    <!-- -->
                </excludes>
                <archive>
                    <manifest>
                        <addClasspath>true</addClasspath>
                        <classpathPrefix>lib/</classpathPrefix>
                        <mainClass>com.w3p.im.iib.mon.client.IIBMonitoringClient</mainClass>
                    </manifest>
                    <manifestEntries>
                        <Class-Path>. resources</Class-Path>
                    </manifestEntries>
                </archive>
            </configuration>
        </plugin>
        
        <!-- Maven Assembly Plugin -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-assembly-plugin</artifactId>
            <version>3.3.0</version>
            <executions>
                <execution>
                    <id>create-archive</id> <!-- this is used for inheritance merges -->
                    <phase>package</phase> <!-- append to the packaging phase. -->
                    <goals>
                        <goal>single</goal> <!-- goals == mojos -->
                    </goals>
                    <configuration>
                        <appendAssemblyId>false</appendAssemblyId>
                        <descriptors>
                            <descriptor>src/main/assembly/zip.xml</descriptor>
                        </descriptors>
                        <outputDirectory>${project.basedir}</outputDirectory>
                    </configuration>
                </execution>
            </executions>
        </plugin>

    </plugins>
</build>

[zip.xml]

<assembly>
<id>zip</id>
<baseDirectory>/</baseDirectory>
<includeBaseDirectory>false</includeBaseDirectory>
<formats>
    <format>zip</format>
</formats>

<fileSets>
    <fileSet>
        <directory>${project.build.directory}/${client.build.dir}_${project.version}</directory>
        <outputDirectory>${client.build.dir}_${project.version}</outputDirectory>
    </fileSet>
</fileSets>

Without seeing your pom files. I would suggest to create the zip file which contains the appropriate structure and not copying files to lib folder for example and afterwards packaging to a zip file... — khmarbaise, Jan 24 '22 at 11:28
Thanks @khmarbaise for a very swift response. I hoped that I could avoid adding the long pom file. I have done that now. Does it help? The immediate dependency is 'com.w3p.njams.client' (aka Proj1). This also has a dependency 'com.w3p.api.iib10' (not mentioned in the general desription). — John Ormerod, Jan 24 '22 at 12:16
To be honest in that pom file there are so many things wrong. Remove all the configuration for outputDirectory from all plugins (go with the convention!) Why are you trying to copy resources in validate phase? That's wrong... Creating a lib directory with maven-dependency-plugin is wrong as well. Changing all that stuff maven-jar-plugin (just keep the defaults). Use the maven-assembly-plugin + an assembly descriptor to describe what you like to achieve...creating a zip file in root of a project is wrong. The `target` directory is the correct location. — khmarbaise, Jan 24 '22 at 13:37
Also plugins as dependencies is wrong too...I strongly recommend a Maven consultancy to support you and save you a lot of work and wrong things... — khmarbaise, Jan 24 '22 at 13:40
@khmarbaise I'm not surprised that I have mistakes. My starting pom code, I created about 3 years ago to build a zip file for a single maven project. I did it using a compination of Maven docs, blog articles, stackOverflow answers and a bi tof native with and cunning. I will go through your suggesstions, RTFM and make one change at a time. Manty thanks — John Ormerod, Jan 24 '22 at 16:47
If you have furthermore questions don't hesitate to ask ...here ... — khmarbaise, Jan 24 '22 at 16:53

Shmulik Klein · Accepted Answer · 2022-01-24T16:12:59.770

-1

It happens because the maven-assembly-plugin executes on a prior phase (package) than the the maven-dependency-plugin phase (install). Try to set up the execution of the plugins so it will act as you expect.

I would also suggest a different approach which I think can simplify you build configuration - use a multi-module pom which will aggregate both project. Than on the concrete pom.xml of AppA use Proj1 as a dependency. It will saves you from copying around files and repackage.

edited Jan 24 '22 at 16:12

answered Jan 24 '22 at 12:44

Shmulik Klein

3,754
19
34

Assembly plugin is explicit bound to the package phase so this statement is simply wrong... – khmarbaise Jan 24 '22 at 13:34
Can you please elaborate why this is wrong? I agree that it is explicitly set to `package` phase which is prior to the explicit `install` phase which is set for the dependency plugin. – Shmulik Klein Jan 24 '22 at 16:14
@ShmulikKlein. Thanks for your suggestions. The reason I abandoned use of a multi-module pom, is because when I want tp build AppB and Proj1, then Proj1 will need 2 parent poms - which is not allowed. That's how I ended up here... – John Ormerod Jan 24 '22 at 16:51
You have a single parent which references Proj1 and Proj2 ... which is not allowed ? Should you solve the problem? – khmarbaise Jan 24 '22 at 16:53
@ShmulikKlein. Thanks very much. It was the phase in the dependency plugin. I could see that I had a comment to say that it hd been 'prepare-package'. I must have changed it a a result of a suggestion from a search. Now it builds the zip file with the jars in the lib folder and in the classpath of the Maniifest. – John Ormerod Jan 24 '22 at 18:10
@ShmulikKlein HOWEVER! I can't find a way to exclude the Maven plugin jar files from being copied to the lib folder. This doesn't happen when the zip file is built from a single project and i can't see anything like an for the plugin jars, From the console: [INFO] Copying maven-dependency-tree-3.0.1.jar to E:\GitRep [INFO] Copying maven-common-artifact-filters-3.2.0.jar to E [INFO] Copying maven-artifact-transfer-0.13.1.jar to E:\Git [INFO] Copying commons-collections4-4.2.jar to E:\GitRepos\ I've searched for an answer and failed to find one. Can you help? – John Ormerod Jan 24 '22 at 18:18
You can use `excludeTransitive` or specific `excludeArtifactIds`, etc. - again, that's probably not the right thing to do. I couldn't understand why the parent pom way is not allowed - there is no need for two parent pom - you only need one parent pom and concrete `pom.xml` for AppB and Proj1. @khmarbaise, can you please elaborate on why did you downvote this answer? – Shmulik Klein Jan 24 '22 at 20:50
1

`I can't find a way to exclude the Maven plugin jar files..`. The first reason there shouldn't be dependencies on plugins at all because that is wrong. It does not even give a hint on that wrong setup... – khmarbaise Jan 24 '22 at 21:02
I agree about that. – Shmulik Klein Jan 24 '22 at 21:10
@ShmulikKlein and khmarbaise. I want to acknowledge you both for staying with me on this. 25 years ago I was tech support for IBM's VisualAge client/server product. I spent quite a lot of my own time helping IBMers and customers on its forum. So, I do know what it's like on your side. Thanks. I spent 3 days last week working through an online Maven by Example course from Nexus to help building using modules and parent pom. – John Ormerod Jan 25 '22 at 10:44
@khmarbaise and Shmulik Klein. In the full picture Proj1 can have one of 2 dependent projects ApiA or ApiB. Thus 2 builds: AppA + Proj1 + ApiA and AppB + Proj1 + ApiB. Proj1 is the 'coode that does all of the work. Currently it can process with ApiA or ApiB. I intended that Proj1, ApiA and ApiB would be modules. As I understand using modules, Proj1 would need to define AppA and AppB as parents. But, Maven only allows Proj1 to have one parent. That's why I moved to using projects. The build works now that I restored the 'prepare=-package' phase to the dependency plugin. – John Ormerod Jan 25 '22 at 10:55
@khmarbaise and Shmulik Klein. Success! I added the Maven plugins as dependencies because it seemed clean and nicely documented the plugins used! Oh well. I have now removed them from Proj1 and ApiA and no Maven jars find their way into the 'lib' folder. You guys have really helped me here and I suddently realised that I no longer feel afraid of Maven! In terms of using modules and a parent pom, would it work to change AppA and AppB from 'project' to 'module' and create a new parents AppA-Parent and AppB-Parent? Otherwise, I am sorted. Best wishes from a very dull and grey UK! – John Ormerod Jan 25 '22 at 12:44
@JohnOrmerod I glad it helped you. Why would you need two different parents? use one parent pom to define the different modules (together with a suitable directory structure) and two concrete poms - one for each project. – Shmulik Klein Jan 27 '22 at 12:20

maven dependencies copied from dependent project after zip assemby

1 Answers1