2

I have a Kafka Streams application. I had no problems with rocksdb or Kafka Streams until we deployed the code to our QA servers. The applications streams state was set to error and this was the error message:

ERROR com.sial.notifications.kafka.streams.AuditStream - Thread: notification-messages-ecf5f120-5cfb-4a2a-91b5-b145e99c3014-StreamThread-1 Uncaught streams exception /tmp/librocksdbjni6472682265799189638.so: /tmp/librocksdbjni6472682265799189638.so: failed to map segment from shared object: Operation not permitted

When the application starts the Rocksdb shared object is written out to /tmp However for security purposes the server mounts /tmp noexec so we cannot run the shared object from /tmp. Is there a way to control where the the so is written?

I found that I could change it by setting java.io.tmpdir system property, but this seems very heavyweight and could have side effects with other packages. Is there a system property that is specific to Rocksdb or Kafka streams that will determine where the so is written?

whomer
  • 575
  • 9
  • 21

1 Answers1

-1

I am also trying to configure where a kafka consumer is storing its stuff and I think you can set a property to do that:

state.dir - Directory location for state stores (default value /tmp/kafka-streams)

So I think that while creating your KafkaStreamsConfiguration, you can inject in it the property: 

StreamsConfig.STATE_DIR_CONFIG

It's also described here

if you want to check the full list of the property you can go here

filmac
  • 177
  • 2
  • 15
  • that is good to know. However it doesn't look like the state.dir property can be used to determine where the RocksDB shard object is executed from. – whomer Jan 31 '19 at 19:29