0

I have a streaming pipeline that uses rocksdbjni 6.15.2 to manage and checkpoint state. I'm trying to use this same library in a separate offline Scala process to read the checkpoint files, and do some further processing.

To test, I copied one of the checkpoint state folders to a local folder:

ls /mnt/tmp/rocksdb/

010959.sst
CURRENT
MANIFEST-010701
OPTIONS-010704

and then ran this simple Scala program:

import org.rocksdb.{Options, RocksDB}

val dbPath = "/mnt/tmp/rocksdb"
val options = new Options().setCreateIfMissing(false)

RocksDB.loadLibrary()
val db: RocksDB = RocksDB.open(options, dbPath)

and run into this exception: org.rocksdb.RocksDBException: Corruption: IO error: No such file or directoryWhile open a file for random read: /mnt/tmp/rocksdb/010958.ldb: No such file or directory

Has anyone run into this Exception before? Why is RocksDB looking for a .ldb file here?

Kevin Kim
  • 1
  • Looking briefly at how Flink opens RocksDB, it appears to set createIfMissing to true; perhaps that's worth a try. FWIW, Flink also uses a different version of open, where it supplies columnFamilyDescriptors and columnFamilyHandles. – David Anderson Mar 25 '21 at 11:19
  • looking for a `.ldb` is fallback when `.sst` file not found, so this exception say that some sst file is missing – mad May 12 '21 at 21:11

0 Answers0