0

I have a Maven project where I have utilized a lot of functionalities from Elasticsearch. More specifically, it was imported in this way:

        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>5.6.10</version>
        </dependency>

Now that I am making some tweaks to Elasticsearch source code in order to get some extra functionality, and I have compiled my code using ./gradlew assemble and have import the compiled jar from ES_SOURCE_CODE_FOLDER/core/build/distributions/elasticsearch-5.6.10-SNAPSHOT.jar in Maven by specifying a systemPath and scope (for now, I know this not what people would actually do but just to test out my implementation):

        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>5.6.10</version>
            <scope>system</scope>
            <systemPath>/Users/hatsuyukisakura/elasticsearch/core/build/distributions/elasticsearch-5.6.10-SNAPSHOT.jar</systemPath>
        </dependency>

However I realize that after I do this, I was unable to compile my program anymore, as the following import was no longer working:

import org.apache.lucene.index.Fields;
import org.apache.lucene.index.TermsEnum;
import org.apache.lucene.queryparser.flexible.standard.QueryParserUtil;

My question is, did I compile Elasticsearch in an incorrect way? How am I suppose to compile it in the same way as it is compiled in the Maven repository, so that I can still get my imports from org.apache.lucene working?

In case if these information are needed: My changes to Elasticsearch code are very limited, based on a branch off 857bfc2ac43ae3986197aeb2177ab5ff87d9f3b4 which still have the 5.6.10 as version number. My working environments are:

$ java -version
openjdk version "1.8.0_212"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_212-b03)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.212-b03, mixed mode)

$ mvn --version
[MVNVM] Using maven: 3.5.2
Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T00:58:13-07:00)
Maven home: /Users/hatsuyukisakura/.mvnvm/apache-maven-3.5.2
Java version: 1.8.0_212, vendor: AdoptOpenJDK
Java home: /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"

$ gradle --version

------------------------------------------------------------
Gradle 4.7
------------------------------------------------------------

Build time:   2018-04-18 09:09:12 UTC
Revision:     b9a962bf70638332300e7f810689cb2febbd4a6c

Groovy:       2.4.12
Ant:          Apache Ant(TM) version 1.9.9 compiled on February 2 2017
JVM:          1.8.0_212 (AdoptOpenJDK 25.212-b03)
OS:           Mac OS X 10.13.6 x86_64
Tamaki Sakura
  • 482
  • 5
  • 22

2 Answers2

1

I can't comment on the way of compiling ElasticSearch from sources, Based on this information it looks like your on track.

However, I can tell, that if after you've come up with a custom distribution, added a dependency in maven and see that org.apache.lucene.index.Fields (for instance) is not found, the best way is to open up the artifact itself with Winzip/Winrar or any other program that can open Zip files (because jar is basically a zip), and make sure that the file org.apache.lucene.index.Fields indeed exists there.

If its there, then something wrong with your maven definitions, and here many things can go wrong, for example, permissions on generated file if you're on linux or maybe clash with other dependency (you can use mvn help:effective-pom to what maven actually decides to pick as a dependency)

If its not there, something went wrong during the Elasticsearch compilation.

Mark Bramnik
  • 39,963
  • 4
  • 57
  • 97
  • Mark, thanks for providing some debugging info, @Tamaki, please see https://stackoverflow.com/questions/43475990/maven-dependency-for-self-placed-jars-in-project-level/43476396#43476396 as this might help you – Amit Aug 26 '19 at 02:00
  • Here's what's strange. I tried to unzip my jar, `org.apache.lucene.index.Fields` is indeed NOT there. But the thing is, for the original jar file I used from maven (https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch/5.6.10), `org.apache.lucene.index.Fields` is also NOT there, but I can import it without any problem! – Tamaki Sakura Aug 26 '19 at 22:42
  • 1
    Since system scope should be non transitive (please veeify this) it wont pick the transitive dependencies. – Mark Bramnik Aug 27 '19 at 05:54
  • hmmm I tried to remove the system scope and install the jar to local maven repo using `mvn install:install-file` but it is still missing the dependency... – Tamaki Sakura Sep 04 '19 at 00:18
  • Install file wont help because it generates the pom without transitive dependencies declaration in your local repository. Hence ghe build system doesn't know which dependencies should be added to the buil process – Mark Bramnik Sep 04 '19 at 05:16
  • Yes you are right - I need to provide pom file manually in the install as what I have done above. – Tamaki Sakura Sep 04 '19 at 20:25
0

@Mark Bramnik 's answer in the follow up section is correct, it is caused by transitive dependencies.

Just remove the system doesn't solve the problem however, I need to do the following:

mvn install:install-file \
   -Dfile=/Users/hatsuyukisakura/elasticsearch/core/build/distributions/elasticsearch-5.6.10-SNAPSHOT.jar \
   -DgroupId=org.elasticsearch \
   -DartifactId=elasticsearch \
   -Dversion=5.6.10-X \
   -Dpackaging=jar \
   -DpomFile=/Users/hatsuyukisakura/elasticsearch/core/build/distributions/elasticsearch-5.6.10-SNAPSHOT.pom

And then use 

        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>5.6.10-X</version>
        </dependency>

Instead

Tamaki Sakura
  • 482
  • 5
  • 22