2

I'm trying to use the Java API for Apache Arrow to connect to a memory store. I've done this in Python, successfully, using the Python API by following the guide here.

I've also looked at the C++ API documentation, but it didn't help much.

The Java Docs makes it look similar to the other documentation.

  1. Make sure the plasma object store is running (usually "/tmp/plasma" for the examples).

    1. Create client
    2. Connect to the client by providing the object store ("/tmp/plasma"), and ("", 0) for the other two parameters.

However, when attempting to use the following line, I get an UnsatisfiedLinkError, that I can't find any reference to within the Apache Arrow documentation. Other solutions found of google (such as calling System.load) haven't been successful either.

PlasmaClient client = new PlasmaClient("/tmp/plasma", "", 0);

A copy of my error messages can be seen below:

Exception in thread "main" java.lang.UnsatisfiedLinkError:org.apache.arrow.plasma.PlasmaClientJNI.connect(Ljava/lang/String;Ljava/lang/String;I)J at org.apache.arrow.plasma.PlasmaClientJNI.connect(Native Method) at org.apache.arrow.plasma.PlasmaClient.<init>(PlasmaClient.java:44) at plas.main(plas.java:11)

Any help is appreciated. Thank you!

aurelius
  • 3,946
  • 7
  • 40
  • 73
SSS
  • 21
  • 2

1 Answers1

2

It's been a while since I was doing this but the issue is quite simple, though not easy to find. In the test routines you can find an important line:

System.loadLibrary("plasma_java");
val plasmaClient = new PlasmaClient("/tmp/plasma","",0);

The shared library libplasma_java.so is created when compiling the C++ extension in CMake with the option -DARROW_PLASMA_JAVA_CLIENT=on. In the arrow/java/plasma/test.sh file you can find the entire build command but this is a derived one from that file:

cd arrow/cpp
mkdir release
cd release
cmake -DCMAKE_BUILD_TYPE=Release \
            -DCMAKE_C_FLAGS="-g -O3" \
            -DCMAKE_CXX_FLAGS="-g -O3" \
            -DARROW_BUILD_TESTS=off \
            -DARROW_HDFS=on \
            -DARROW_BOOST_USE_SHARED=on \
            -DARROW_PYTHON=on \
            -DARROW_PLASMA=on \
            -DPLASMA_PYTHON=on \
            -DARROW_JEMALLOC=off \
            -DARROW_WITH_BROTLI=off \
            -DARROW_WITH_LZ4=off \
            -DARROW_WITH_ZLIB=off \
            -DARROW_WITH_ZSTD=off \
            -DARROW_PLASMA_JAVA_CLIENT=on \
            ..
make VERBOSE=1 -j4
make install

I just ran it and it built everythin as desired. If you want to install it somewhere simply add a prefix -DCMAKE_INSTALL_PREFIX=/home/myuser/install-here.

If you run System.loadLibrary("plasma_java") it only works if the .so can be found somewhere on your system (such stuff is usually specified on Linux by the environment variable LD_LIBRARY_PATH). Note that this also requires libplasma.so and libarrow.so in the system path as well.

codie
  • 343
  • 4
  • 12