Publishing to and resolving from Local Ivy Cache by Example

Question

I think the most astonishing thing here is that this functionality is baked in by default to Maven and Gradle, yet, there are no traces of its existence in the Ant/Ivy landscape (see for yourself!).

---

I have inherited a suite of JVM components that use Ant/Ivy as their build/dependency system. There are lots of dependencies between each of these components, which means making a change to one of them usually has a ripple effect requiring you to update Ivy dependencies and publish new version of upstream dependencies.

The old team maintaining these projects handled local development by publishing snapshot jars to a snapshot repo. I would like to replace this paradigm with a new one whereby snapshots are published to/resolved from the local Ivy cache.

I was able to find this very similar question but found the answer a bit lacking on details (particularly a fully stitched-together code snippet), in part because the question lacked any specific code examples. So I have created an SSCCE here and have pushed 2 GitHub repos:

• fizzbuzz-model, a Java library that defines a data model (some meaningless POJOs)
• fizzbuzz-app, a simple executable jar that depends on fizzbuzz-model as a dependency

What I am looking for here are the exact (that is, actual code, not pseudo-code) changes (likely to build.xml, ivy.xml or ivy-settings.xml, or all three!) that will allow me to use the following local dev/test cycle:

1. I make a change to fizzbuz-model and publish the change locally to the Ivy cache, preferably as a snapshotted version (such as 1.0.0-SNAPSHOT or similar)
2. From inside of the fizzbuzz-app root directory, I run ant resolve which pulls in those changes from the cached snapshot
3. Now I can make use of those changes in fizzbuzz-app

Though not a hard requirement, I would ideally like to not have to manage version numbers manually. That is, when I publish fizzbuzz-model locally, it overwrites the current binary with the same version (again, like fizzbuz-model-1.0.0-SNAPSHOT.jar) instead of incrementing the buildnumber to, say, fizzbuzz-model-1.0.1-SNAPSHOT.jar or similar). That way all I have to do when testing locally is publish fizzbuzz-model and resolve fizzbuzz-app.

Currently, when I publish fizzbuzz-model, I get the following errors:

/Users/myuser/workspace/fizzbuzz-model/build.xml:52: impossible to publish artifacts for hotmeatballsoup#fizzbuzz-model;1.0: java.io.IOException: missing artifact hotmeatballsoup#fizzbuzz-model;1.0.0-SNAPSHOT!fizzbuzz-model.pom
    at org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:225)
    at org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:172)

To reproduce locally, clone both those projects and follow their READMEs, starting with fizzbuzz-model. Can anyone spot where I'm going awry? Feel free to answer here and/or submit a PR, whichever you prefer! And thanks!

Answer 1

The error indicates that ivy cannot find a file to be published, in the local build workspace. It is not a snapshot issue.

The problem is here:

<ivy:publish resolver="local" pubrevision="1.0.0-SNAPSHOT" >
  <artifacts pattern="dist/[artifact]-[revision].[ext]" />
</ivy:publish>

You have included a "revision" in the pattern but unfortunately the jar you created locally does not match this naming convention. It's missing both the revision and thereis a typo (should be "fizzbuzz", not "fizbuz"):

<target name="dist" depends="clean,compile">
  <jar jarfile="dist/fizzbuz-model.jar" basedir="build/main" />
</target>

I predict further issues because you are attempting to configure ivy to emulate the way Maven stores snapshot revisions. This requires additional metadata files that have never been officially documented by Maven. I would highly recommend pushing your published file to a Maven repository manager for the correctly formatted SNAPSHOT storage.

The following are examples of publishing artefacts to a Maven repo from ivy

• Publish SNAPSHOT artifacts to Maven using IVY - what's the magic?
• https://support.sonatype.com/hc/en-us/articles/213465388-How-do-I-configure-my-Ivy-build-to-deploy-artifacts-to-Nexus-

---

Suggested solution

I have submitted to Pull requests as requested with my suggestion on how a local ivy repository can be used in preference to snapshot releases:

• fizzbuzz-model - (Pull request)
• fizzbuzz-app - (Pull request)

In summary the model publishes a new revision with every fresh build

<target name="publish" depends="clean,dist">

    <!-- Determine build number from previously published revisions -->
    <ivy:buildnumber resolver="local" organisation="${ivy.organisation}" module="${ivy.module}" revision="${target.release}"/>

    <!-- Resolve ivy dependencies and create a Maven POM file -->
    <ivy:deliver deliverpattern="dist/ivy.xml" pubrevision="${ivy.new.revision}" status="release"/>
    <ivy:makepom ivyfile="dist/ivy.xml" pomfile="dist/fizzbuzz-model.pom" />

    <!-- Publish the local repo. Defaults to ~/.ivy2/local -->
    <ivy:publish resolver="local" pubrevision="${ivy.new.revision}" >
        <artifacts pattern="dist/[artifact].[ext]" />
    </ivy:publish>
</target>

And this is used as a dynamic dependency in the app code, always retrieving the lastest published release of the other module.

<dependencies>
    <dependency org="hotmeatballsoup" name="fizzbuzz-model" rev="latest.integration" conf="compile->default" />
</dependencies>

Notes:

• As a bonus a POM file is also being generated, strictly speaking it's not necessary unless you're pushing artifacts to a Maven repo.

---

Background

Dependency managers like ivy come from a time before Maven's universal popularity (I remember when Maven 1.0 was universally hated), so it's unsurprising that ivy doesn't exactly implement the Maven's workflow.

Maven is and always has been a highly opinionated tool. Confusingly it supports two ways to publish an artifact. As a release or as a snapshot. In my opinion this is what causes the most friction when attempting to use Maven for continuous deployment, where all releases should be considered for release. But it cannot be denied that Maven is the universal way to distribute all Java based binaries. Maven repositories can serve as a single integration point supporting multiple build technologies, Maven, Gradle or ANT/Ivy.

So one needs to first accept that the snapshot development workflow is very peculiar to Maven, and to the best of my knowledge has not been replicated by any other repository formats.

In other repositories the version number for a release is always unique. That applies to official releases, candidate release or development releases. A Maven snapshot, on the other hand, is never finalized. What do I mean by that? If I build against version "1.0-SNAPSHOT" today, the dependency could be completely different tomorrow. That is because each new snapshot build will either create a new timestamped artefact behind the scenes to just overwrite the previously stored binary.

Ivy has a different mechanism for supporting development builds (Different does not mean better). One can depend on the latest version in development, but this will always be explicitly resolved at build time. When one publishes an artefact to an ivy repository then the handy deliver task is capable of creating a fully resolved ivy file. There is never any ambiguity about what version of a dependency was used.

So in conclusion, first make a call on whether you actually need to support a snapshot workflow. Unless you intend to integrate with other teams using Maven it is unwise to force ivy to follow a workflow it was not designed to support.