Reverse iteration indexed by prefix RocksDb

Question

I'm currently using a RocksDB database for a personal project and I try to get all the content in reverse order of my database which is indexed by a prefix. However I don't manage to find the good code for my loop. I fell like I'm missing something. Here is my code :

    //My iterator is already created with the rocksdb::ReadOptions()
    rocksdb::Slice key{reinterpret_cast<const char *>(indexToFind), sizeof(indexToFind};
    std::vector<int> ids;

    it->SeekForPrev(key);
    auto currentKey = it->key();
    while(it->Valid() && it->key().starts_with(key)) {

        /* Some custom treatment here */

        it->SeekForPrev(it->key());
        it->Prev();
        currentKey = it->key();
    }

    if (!it->status().ok()) {
        std::cout << it->status().ToString() << std::endl;
    }
    assert(it->status().ok()); // Check for any errors found during the scan

Cheers and thanks in advance, Clément.

On opening the database you could add an self-written ByteWiseComparator, which is used by rockdb to sort desc instead of asc. — Zelldon, Nov 06 '18 at 10:02

score 0 · Answer 1 · answered Sep 24 '18 at 07:17

I dont think Reverse iteration using Prefix is supported yet by native rocksdb. You can very well find it from their wiki for Prefix Seek APIs. The following is the limitation of PrefixSeekAPI from the wiki,

SeekToLast() is not supported well with prefix iterating. SeekToFirst() is only supported by some configurations. You should use total order mode, if you will execute those types of queries against your iterator.

One common bug of using prefix iterating is to use prefix mode to iterate in reverse order. But it is not yet supported. If reverse iterating is your common query pattern, you can reorder the data to turn your iterating order to be forward. You can do it through implementing a customized comparator, or encode your key in a different way.

Refer to the wiki here

score 0 · Answer 2 · answered Jun 05 '23 at 09:27

public List<byte[]> reverseList(int limit, byte[] prefixKey, byte[] startKey, byte[] endKey) {
    ReadOptions options = new ReadOptions();
    options.setPrefixSameAsStart(true);
    options.setTotalOrderSeek(true);
    RocksIterator iterator = rocksDB.newIterator(options);
    List<byte[]> result = new ArrayList<>();
    for(iterator.seekForPrev(getNextByteArray(prefixKey));iterator.isValid(); iterator.prev()){
        byte[] key = iterator.key();
        result.add(key);
    }
    iterator.close();
    return result;
}

public static byte[] getNextByteArray(byte[] bytes) {
    byte[] result = new byte[bytes.length];
    int carry = 1;
    for (int i = bytes.length - 1; i >= 0; i--) {
        int sum = (bytes[i] & 0xff) + carry;
        result[i] = (byte) sum;
        carry = (sum >> 8) & 0xff;
    }
    return result;
}

Reverse iteration indexed by prefix RocksDb

2 Answers2