0

I have created one kafka streaming application. In that, I am trying to aggregate a KStream into KTable. There, i have specified one statestore which is used internally by Kafka. However, on running the application, I am getting below error.

Exception in thread "StreamThread-1" java.lang.UnsatisfiedLinkError: /tmp/librocksdbjni858257496864179953.so (  0509-022 Cannot load module /tmp/librocksdbjni858257496864179953.so.
    0509-026 System error: Cannot run a file that does not have a valid format.)
    at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1213)
    at java.lang.System.load(System.java:517)
    at org.rocksdb.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:78)
    at org.rocksdb.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:56)
    at org.rocksdb.RocksDB.loadLibrary(RocksDB.java:64)
    at org.rocksdb.RocksDB.<clinit>(RocksDB.java:35)
    at java.lang.J9VMInternals.initializeImpl(Native Method)
    at java.lang.J9VMInternals.initialize(J9VMInternals.java:236)
    at org.rocksdb.Options.<clinit>(Options.java:22)
    at java.lang.J9VMInternals.initializeImpl(Native Method)
    at java.lang.J9VMInternals.initialize(J9VMInternals.java:236)
    at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:115)
    at org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:148)
    at org.apache.kafka.streams.state.internals.ChangeLoggingKeyValueBytesStore.init(ChangeLoggingKeyValueBytesStore.java:39)
    at org.apache.kafka.streams.state.internals.MeteredKeyValueStore$7.run(MeteredKeyValueStore.java:100)
    at org.apache.kafka.streams.processor.internals.StreamsMetricsImpl.measureLatencyNs(StreamsMetricsImpl.java:188)
    at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:131)
    at org.apache.kafka.streams.state.internals.CachingKeyValueStore.init(CachingKeyValueStore.java:62)
    at org.apache.kafka.streams.processor.internals.AbstractTask.initializeStateStores(AbstractTask.java:86)
    at org.apache.kafka.streams.processor.internals.StreamTask.<init>(StreamTask.java:141)
    at org.apache.kafka.streams.processor.internals.StreamThread.createStreamTask(StreamThread.java:834)
    at org.apache.kafka.streams.processor.internals.StreamThread$TaskCreator.createTask(StreamThread.java:1207)
    at org.apache.kafka.streams.processor.internals.StreamThread$AbstractTaskCreator.retryWithBackoff(StreamThread.java:1180)
    at org.apache.kafka.streams.processor.internals.StreamThread.addStreamTasks(StreamThread.java:937)
    at org.apache.kafka.streams.processor.internals.StreamThread.access$500(StreamThread.java:69)
    at org.apache.kafka.streams.processor.internals.StreamThread$1.onPartitionsAssigned(StreamThread.java:236)
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:255)
    at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:339)
    at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:303)
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:286)
    at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1030)
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:995)
    at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:582)
    at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:368)

Kafka version: 10.2.0

Manish Bansal
  • 2,400
  • 2
  • 21
  • 37

1 Answers1

0

This has been addressed here - https://github.com/facebook/rocksdb/issues/2071

We don't provide builds of the native component for AIX in the Jar file. As you are on AIX you will need to build your own.

adamretter
  • 3,885
  • 2
  • 23
  • 43
  • I don't understand why i need to re-build this. I downloaded the binaries which already has this jar pre-packaged. Kindly enlighten me. Thank you in advance. – Manish Bansal Mar 31 '17 at 14:17
  • 1
    Link only answers are considered bad practice at SO. You should rather copy the relevant content into your answer and add the link as reference only. – Matthias J. Sax Mar 31 '17 at 17:39
  • The prebuild binaries are not compatible with AIX platform. Note, Kafka Stream officially only supports Linux and even if AIX is a Unix based system, it's often not compatible and you need different binaries. – Matthias J. Sax Mar 31 '17 at 17:39
  • All right. I will try to re-build on Monday and update the forum. – Manish Bansal Apr 01 '17 at 02:54
  • @adamretter I tried re-building rocksdbjava. But i am getting errors related to no such file or directory like " ./include/rocksdb/db.h:17:25: error: unordered_map: No such file or directory". Any idea? I am using GNU make 3.80 and having gcc 4.1.0 – Manish Bansal Apr 03 '17 at 08:58
  • Nope. I would suggest opening a GitHub issue – adamretter Apr 04 '17 at 20:09