2

I was learning flatbuffers from this link , there was no example to demonstrate how to store dictionary(map). There was a mention of "Storing dictionaries in java/Csharp" in this link , but i did not understand much about it. I am from java background. Any example of how to store dictionary/map in flatbuffers in java would be helpful

stallion
  • 1,901
  • 9
  • 33
  • 52
  • https://google.github.io/flatbuffers/flexbuffers.html – Shivendra Agarwal Jul 20 '18 at 10:06
  • flexbuffers will be efficient to store dicts.. see the example in link above – Shivendra Agarwal Jul 20 '18 at 10:07
  • Thanks @ShivendraAgarwal.. I went through it and i am still confused because there is no schema for that and i am not C guy. Do you have any example you can share in java along with schema for map in flexbuffer.. this would help me a lot, Thanks Vinay S – stallion Jul 20 '18 at 10:41
  • I recommend you follow what it says under "Storing dictionaries in FlatBuffers". Have you tried the steps? where do you get stuck? Using FlexBuffers like Shivendra says will not work, since those are not available in Java. – Aardappel Jul 20 '18 at 14:38
  • Flexbuffers are now available in Java, see [FlatBuffers release 1.12.0](https://github.com/google/flatbuffers/releases/tag/v1.12.0) – mfunaro Jan 08 '21 at 17:53

1 Answers1

0

I realize this is an old question but I came across it when I was trying to figure out the same thing. Here is what I did to get a "dictionary/map"

Schema File

namespace com.dictionary;

table DataItem{
  keyValue:string(key);
  name:string;
  age:int;
  codes[string];
}

table DictionaryRoot{
  items:[DataItem];
}

root_type DictionaryRoot;

When you run this through the FlatBuffers compiler flatc -j schema.fbs it will produce two Java files, one named DictionaryRoot.java, the other named DataItem.java.

In your Java application

Using those two generated Java files you will need to construct the buffer. This has to be done from the innermost data to the outermost. So you need to construct your DataItems (and keep track of their offsets) before your DictionaryRoot.

In this example, let's assume that you have a map of Objects in Java that you need to create the buffer from.

List<Integer> offsets = new ArrayList<>();
FlatBufferBuilder builder = new FlatBufferBuilder(1024);

for (Entry<String, DataObj> entry : map.entrySet()) {
  DataObj dataObj = entry.getValue();

  // use the builder to create the data and keep track of their offsets
  int keyValueOffset = builder.createString(entry.getKey());
  int nameOffset = builder.createString(dataObj.getName());
  int ageOffset = dataObj.getAge();
  int[] codesOffsets = dataObj.getCodes().stream().mapToInt(builder::createString)
      .toArray();

  // use the builder to create a vector using the offsets from above
  int codesVectorOffset = DataItem.createCodesVector(builder, codesOffsets);

  // now with all the inner data offsets, create the DataItem
  DataItem.startDataItem(builder);

  DataItem.addKeyValue(builder, keyValueOffset);
  DataItem.addName(builder, nameOffset);
  DataItem.addAge(builder, ageOffset);
  DataItem.addCodes(builder, codesVectorOffset);

  // ensure you 'end' the DataItem to get the offset
  int dataItemOffset = DataItem.endDataItem(builder);

  // track the offsets
  offsets.add(dataItemOffset);
}

// use the builder to create a sorted vector from your offsets. This is critical
int sortedVectorOffset = builder.createSortedVectorOfTables(new DataItem(),
    offsets.stream().mapToInt(Integer::intValue).toArray());

// now with your sorted vector, create the DictionaryRoot
DictionaryRoot.startDictionaryRoot(builder);

DictionaryRoot.addItems(builder, sortedVectorOffset);

int dictRootOffset = DictionaryRoot.endDictionaryRoot(builder);

// signal to the builder you are done
builder.finish(dictRootOffset);

// Write data to file
FileOutputStream outputStream = new FileOutputStream("output.bin");
outputStream.write(builder.sizedByteArray());
outputStream.close();

I hope that will help someone else along their journey using FlatBuffers.

mfunaro
  • 574
  • 6
  • 23