Ivy can not resolve the scope of a dependency which is a dependency of a transitive dependency

Question

I add a dependency(let's name it as A) to ivy.xml which has a pom file in maven central. Ivy uses ibiblio for resolving the maven dependencies. The dependency(A) which is added to ivy.xml has a transitive dependency(B). So far so good till here. The dependency(C) of transitive dependency(B) can not be resolved by ivy.

I defined A in ivy.xml like this:

<dependency org="Z" name="A" rev="0.6-SNAPSHOT" conf="*->default"/>

In pom file of B, C is defined both in compile and test scopes like below:

<dependency>
      <groupId>X</groupId>
      <artifactId>C</artifactId>
    </dependency>
    <dependency>
      <groupId>X</groupId>
      <artifactId>C</artifactId>
      <type>test-jar</type>
      <scope>test</scope>
</dependency>

When I look the xml file of B which is resolved by ivy in ivy's cache file(~/.ivy2/cache/X/C/ivy-0.98.8-hadoop2.xml), it looks like this:

<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)"/>
<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)">
  <artifact name="C" type="test-jar" ext="jar" conf="" m:classifier="tests"/>
</dependency>

For this reason, ivy can not define C scopes correctly. For the record, I don't have permissions to modify the pom files as they are third party projects. How can I fix it ?

Answer 1

I reviewed the ivy usage of the nutch project and apologies but my conclusion is that it's overly complex for the following reasons:

• "compile" and "test" targets are issuing separate calls to the resolve task
• Each plugin is also calling an ivy resolve task
• Complex logic for maintaining classpaths. Could be simplified using the cachepath task and ivy configurations.
• Build plugins are not managed by ivy (Sonar, eclipse, rat)

I started to refactor the build, but had to stop when I realised that I didn't understand the relationship between the main nutch artifact and the plugins... (I discovered NUTCH-1515 the hard way... big time-waster The feed plugin has missing dependencies).

I also noticed issue NUTCH-1371 calling for the removal of ivy. This would be a tricky refactoring without significant change to the current codebase. I suspect it would have to be a multi-module build with each plugin listing its own dependencies.

In conclusion, this work does not answer your question, but thought I needed to at least document the result of a few hours analysis :-) In light of NUTCH-1371 I don't know if your project will tolerant major ivy refactoring?

Refactoring ivy

Here follows what I achieved so far:

• Private "development" fork of the nutch project
• Diff with trunk

Benefits:

• Single ivy report showing all configurations (New ivy-resolve target)
• New mechanism for installing ivy (New ivy-install target)
• Classpaths are managed using ivy configurations (See use of ivy cachepath task and configurations in ivy file)
• Eclipse, sonar and rat ANT tasks automatically installed using ivy (The Eclipse plugin is noteworthy as it uses a packager resolver to download and extract jar from a tar archive).

Impacts the following Nutch issues

• NUTCH-1881 : This new approach removes resolve-test and resolve-default targets and manages the classpaths using ivy instead of the ${build.lib.dir}
• NUTCH-1805 : Can easily setup a separate configuration for the job target with it's own dependencies.
• NUTCH-1755 : I think this one is fixed by assigning a name to the the build.xml (see: diff)