I'm looking for an in-memory binary format for a map structure (key-value store) with quick access time.
My goal is to:
- Store this binary format as a file.
- Be able to retrieve it and with zero-copy, using mmap
- Be able to search it, with speed similar to a native map structure.
- As the file will be open using mmap, I want to be able open it from several process using the same memory footprint
Ideally I'd want this binary format to work in several languages (Python, Golang, ...).
Here the example format I'm looking for: {string: string}
E.g. { "Entity_1": "json encoded string" "Entity_2": "json encoded string" ... "Entity_3": "json encoded string" }
I'm trying to find what Apache Arrow did for columnar store, but for key-value store :)
I looked at zero-copy structures like FlatBuffers and Captnproto but they don't supports maps.