3

RocksDb: Multiple values per key (c++)

what i am trying to do

I am trying to adapt my simple blockchain implementation to save the blockchain to the hard drive periodically and so i looked info different db solutions. i decided to use RocksDb due to its ease of use and good documentation & examples. i read through the documentation and could not figure out how to adapt it to my use case. i have a class Block `

class Block {
public:
    string PrevHash;
    
private:
    blockheader header; // The header of the block 
    uint32_t index; // height of this block
    std::vector<tx_data> transactions; // All transactions in the block in a vector
    std::string hash; // The hash of the block
    uint64_t timestamp; // The timestamp this block was created by the node 
    std::string data; // Extra data that can be appended to blocks (for example text or a smart contract)
                      // - The larger this feild the higher the fee and the max size is defined in config.h
};

which contains a few variables and a vector of a struct tx_data. i want to load this data into a rocksdb database.

what i have tried

after google failed to return any results on storing multiple values with one keypair i decided i would have to just enclose each block data in 0xa1 at the beginning then at the end 0x2a

*0x2a*
header
index
txns
hash
timestamp
data
*0x2a*

but decided there was surely a simpler way. I tried looking at the code used by turtlecoin, a currency that uses rocksdb for its database but the code there is practically indecipherable, i have heard about serialization but there seems to be little info out there on it.

perhaps i am misunderstanding the use of a DB?

Community
  • 1
  • 1
Ariel Hurdle
  • 68
  • 1
  • 13

2 Answers2

2

You need to serialization it. Serialization is the process of taking a structured set of data and making it into one string, number or vector of bytes that can then be de-serialized later on back into that struct. One method would be to take the hash of the block and use it as the key in the db then crate a new struct which does not contain the hash. Then write a function that takes a Block struct and a path and constructs a BlockNoHash struct and saves it. Then another function to read a block from a hash and spit out a Block Struct. Very basically you could split each field with a charector which will never occur in the data (eg ` or |), though this means if one piece of the data is corrupted then you cant get any of the other data

Leah Cornelius
  • 495
  • 2
  • 9
1

There are two related questions here.

One is: how do you store complex data -- more than just a simple integer or string -- within a key-value store like RocksDB. As Leo says, you need to serialize them.

Rather than writing your own code, the typical easier way is to use a framework like Protobuf or Thrift to generate code to translate between your in-memory structures and a flat bytes representation suitable to store in a database (or send over the network.)

A related question, from the title: how do you store multiple values per key?

There are two main options:

  • Use a compound key, that distinguishes the various values. By walking a key prefix you can find all the values in a set of related keys. This is better if the values get very large or if you want to find and update them independently.

  • Or, make the value for a single key actually be a compound object that includes several inner values. This is easiest if you always want to fetch all the sub-values in a single operation.

poolie
  • 9,289
  • 1
  • 47
  • 74