Add content of a vector into a Capnproto map object

Question

As answered in this question:

Note that an individual List value in a Cap'n Proto structure has a limit of 2^29-1 elements

Due to this limitation for a single list, I'm trying to split one list that contains more than this amount of items into a map of lists. To do it, I'm using the following schema:

struct Map64UInt{
    entries @0: List(Entry);
    struct Entry{
        key @0: UInt64;
        value @1: List(UInt64);
    }
}

I've been looking into all examples of the Cap'n Proto but I couldn't find one that contains an example on how to create and add itens into a Capn'Proto List, then add this list into a Cap'n Proto Map. As an example, consider the following code:

void addVecToCapnprotoMap(std::vector<uint64_t> &v){
    unsigned int key = 0;
    //Here: how do I create a Capn' Proto List of uint64_t
    //and add it to a Capn Proto map that follows the schema
    //described above?
}

Answer 1

Unfortunately, you cannot dynamically add new elements to a list; you have to specify the full size of the list when you first create it. This is a side effect of Cap'n Proto's zero-copy nature. Since the list is allocated in-place directly into the final message buffer, it can't be resized later.

What you can do instead is maintain a map separately and then write the final list as a last step, like:

// WARNING: Not tested, probably has typos.
class MapHelper {
public:
  MapHelper(Map64UInt::Builder finalBuilder)
      : finalBuilder(finalBuilder),
        orphanage(capnp::Orphanage::getForMessageContaining(finalBuilder)) {}

  void add(uint64_t key, const std::vector<unit64_t>& v) {
    auto orphan = orphanage.newOrphan<capnp::List<uint64_t>>(v.size());
    auto listBuilder = orphan.get();
    for (size_t i = 0; i < v.size(); i++) {
      listBuilder[i] = v[i];
    }
    contents.insert(std::make_pair(key, kj::mv(orphan)));
  }

  void finish() {
    // Write all entries to the final map.
    auto entriesBuilder = finalBuilder.initEntries(contents.size());
    size_t i = 0;
    for (auto& entry: contents) {
      auto entryBuilder = entriesBuilder[i];
      entryBuilder.setKey(entry.first);
      entryBuilder.adoptValue(kj::mv(entry.second));
      ++i;
    }
  }

private:
  Map64UInt::Builder finalBuilder;
  capnp::Orphanage orphanage;
  std::map<uint64_t, capnp::Orphan<capnp::List<uint64_t>>> contents;
};