1

I'm currently working on a project that uses (among others) nd4j, deeplearning4j dependencies. These dependencies include versions for different OS, like "nd4j-native-1.0.0-beta6-android-arm.jar" or "nd4j-native-1.0.0-beta6-linux-armhf.jar" etc.

As the project will only run on linux-x64, I'm wondering whether there's a way to tell Maven to not copy the JARs that are not needed. I hope to reduce the required disk space needed by unnecessary libs.

The pom.xml currently looks like this to be able to produce a deployable output:

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>MyGroup</groupId>
    <artifactId>MyApp</artifactId>
    <version>1.0-SNAPSHOT</version>
    <properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.eclipse.paho</groupId>
            <artifactId>org.eclipse.paho.client.mqttv3</artifactId>
            <version>1.2.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.clerezza.ext</groupId>
            <artifactId>org.json.simple</artifactId>
            <version>0.4</version>
        </dependency>
        <dependency>
            <groupId>com.kohlschutter.junixsocket</groupId>
            <artifactId>junixsocket-core</artifactId>
            <version>2.3.1</version>
        </dependency>
        <dependency>
            <groupId>com.kohlschutter.junixsocket</groupId>
            <artifactId>junixsocket-server</artifactId>
            <version>2.3.1</version>
        </dependency>
        <dependency>
            <groupId>org.nd4j</groupId>
            <artifactId>nd4j-native-platform</artifactId>
            <version>1.0.0-beta6</version>
        </dependency>
        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>deeplearning4j-core</artifactId>
            <version>1.0.0-beta6</version>
        </dependency>
        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>deeplearning4j-parallel-wrapper</artifactId>
            <version>1.0.0-beta6</version>
        </dependency>
    </dependencies>

    <packaging>jar</packaging>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <executions>
                    <execution>
                        <id>copy-dependencies</id>
                        <phase>prepare-package</phase>
                        <goals>
                            <goal>copy-dependencies</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${project.build.directory}/libs</outputDirectory>
                            <useRepositoryLayout>true</useRepositoryLayout>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <configuration>
                    <archive>
                        <manifest>
                            <addClasspath>true</addClasspath>
                            <classpathLayoutType>repository</classpathLayoutType>
                            <classpathPrefix>libs/</classpathPrefix>
                            <mainClass>MyMainClass</mainClass>
                        </manifest>
                    </archive>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>
Max Senft
  • 610
  • 4
  • 13
  • Hmhm, I just found this topic: https://stackoverflow.com/q/34839656/2086717. Seems to be similar but the answer is not what I was looking for. The proposed filter mechanism by the topic creator though is what would be nice to have if there's no other way to accomplish the task. – Max Senft Feb 21 '20 at 10:52

2 Answers2

4

Generally if you want to bundle platform specific binaries for dl4j or anything based on javacpp, you can do the following: https://github.com/bytedeco/javacpp-presets/wiki/Reducing-the-Number-of-Dependencies

Concretely, you would just specify -Djavacpp.platform=linux-x86_64 for all javacpp dependencies.

Adam Gibson
  • 3,055
  • 1
  • 10
  • 12
  • This might be the solution I'm looking for! :-D One further question: I'm trying to put that setting into my pom.xml, but if I add linux-x86-64 into the ... section, it won't work. If I add the -D... option to the command line, it works ... – Max Senft Feb 21 '20 at 11:58
  • I added an issue over at the JavaCPP GitHub (https://github.com/bytedeco/javacpp-presets/issues/846), as I'm currently not 100% sure where the -D / issue acutally lies. – Max Senft Feb 21 '20 at 15:02
1

You can set <exclusions> on dependencies:

https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html

An example would be:

   <dependency>
      <groupId>com.something</groupId>
      <artifactId>some-jar</artifactId>
      <version>1.0.0</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>  
          <groupId>de.dumm</groupId>
          <artifactId>nicht-benutzen</artifactId>
        </exclusion>
      </exclusions> 
    </dependency>
J Fabian Meier
  • 33,516
  • 10
  • 64
  • 142