The the dependencies from an Uber jar at runtime from another jar

Question

Problem: My Uber Jar / Fat jar is too large. The dependencies contained make my jar size over 20 MB. As a result, it can often take several seconds to upload files to the test VPS I am developing on. My objective is to reduce the the size of my jar file so I can quickly upload and test my code. My code itself is only around 500 kb. Its because of the dependencies I need that the jar size is 20 MB.

Attempted Solution: I know that in gradle, the compileOnly scope is for "Dependencies whose API is required at compile time but whose implementation is to be provided by a consuming library, application or runtime environment." So I thought to experiment with creating an empty uber jar with an empty main class, but one that has all the dependencies my project uses. Then, the idea was I would use this uber jar at runtime to access the dependencies I need for my project from with in my actual code jar.

Problem: Now the problem is, I have no idea how to provide the "implementation" as with in this uber jar to my "thin jar" containing MY code during runtime. How do I approach this problem? I know its possible, because from my background in Minecraft plugin development, I know the "implementation" of the methods in other plugin jars are indeed provided during runtime. That allows plugin jar sizes to be smaller.

Have you considered using a version control tool like git, a build tool like maven and having the compile and deploy on your vps be from a local copy of the git repo? You could even add another vps instance and setup [CircleCI](https://circleci.com/) (or another Continuous Integration solution). — Elliott Frisch, Aug 14 '21 at 22:29
I am already using git. But the issue isn't git or deploying to my VPS. The issue is time. It takes too long to deploy a jar that has 20 mb packed into it. I am searching for a solution to reduce my jar size so I can rapidly deploy and test my code. — SkylixMC, Aug 14 '21 at 22:32
the answer you don't want; use spring boot, they solved the fat jar issue. — DwB, Aug 14 '21 at 22:34
You caught only part of my message; I'm saying build the jar on the VPS. That way only the text of your code needs to be transmitted. And `git` is perfect for that. Clone your repo onto the vps. — Elliott Frisch, Aug 14 '21 at 22:34
Why are you building a fat jar at all? if you built a traditional jar + dependencies, then you could easily rsync the result and it would be relatively compact. — Joachim Sauer, Aug 14 '21 at 22:39
With out the fat jar, although gradle compiles the project file, I get ClassNotFoundExceptions during runtime. Perhaps its my inexperience, but shadowjar is the only way I knew to fix that. — SkylixMC, Aug 14 '21 at 22:43
@DwB I am interested in your reply. What is spring boot and how does it solve this issue? — SkylixMC, Aug 14 '21 at 22:54
Spring boot includes the dependencies in the jar file, but it is not a fat jar. — DwB, Aug 15 '21 at 01:51

score 0 · Answer 1 · answered Aug 15 '21 at 01:51

I have found the solution to this problem. As it turns out, I was indeed on the correct track. If like me, your programming efforts are being bottlenecked by a huge jar size, then keep reading.

Once again, to clarify, the approach here is to first create an empty uber jar with no code that contains all your libraries and dependencies. Then, you use that uber jar as a weightless runtime library from a "thin" jar.

Here is a step by step on how to do this:

Use shadowJar on gradle to create an uber jar. Use the "api" scope using gradle's "java-library" plugin. This will make sure that when you use this uber jar as a dependency, you will be able to access all the libraries the uber jar is implementing. Here is an example build.gradle:

plugins {
    id "java"
    id "java-library"
    id "com.github.johnrengelman.shadow"
}

group 'my.company'
version '1.0.0'

repositories {
    maven {
        url 'https://jitpack.io'
    }
    maven {
        url 'https://jcenter.bintray.com'
    }
}

dependencies {
    api 'com.fasterxml.jackson.core:jackson-core:2.10.0'
    api 'com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.9.8'
    api 'com.fasterxml.jackson.core:jackson-databind:2.10.0'
    api 'org.apache.httpcomponents:httpclient:4.5.6'
    api 'org.apache.httpcomponents:httpmime:4.5.6'
    api 'com.zaxxer:HikariCP:2.6.0'
    api 'mysql:mysql-connector-java:5.1.36'
}

shadowJar {
    archiveFileName = "Uber.jar"
    destinationDirectory = file("build")
}

// Force character encoding in case the workspace was not set up correctly
tasks.withType(Javadoc) {
    options.encoding = 'UTF-8'
}

Now upload this uber jar to maven, or where ever you want in order to add the uber jar as a dependency in your main project. Make sure you add this uber jar as a "compileOnly" dependency. Here is the modified build.gradle on my main project.

plugins { id "java" id "com.github.johnrengelman.shadow" }

group 'my.company' version '2.0'

repositories { maven { url 'https://jitpack.io' } maven { url 'https://jcenter.bintray.com' } }

dependencies { compileOnly 'my.company:uberjar:1.0.0' }

shadowJar { archiveFileName = project.name + ".jar" destinationDirectory = file("build") }

jar { manifest { attributes 'Main-Class': 'Main' // your main class attributes 'Class-Path': 'Uber.jar' // path to the uber jar - in this case its in the same directory as the main jar. } }

// Force character encoding in case the workspace was not set up correctly tasks.withType(Javadoc) { options.encoding = 'UTF-8' }

As shown above in my build.gradle, the real magic happens by configuring the path to the uber jar in the "Class-Path" option.
Compile.
Now you can leave your uber jar with its libraries in place and have fun debugging the short and sweet "thin" jar thats just your own code :D!

The the dependencies from an Uber jar at runtime from another jar

1 Answers1