1

I want to access mapdb maps straight from HDFS, without copying it to local file system. Is it possible? Is there is any other approach by which I can read these maps without using local file system?

Akash Mahajan
  • 512
  • 4
  • 16
  • Since HDFS stores files in blocks, unless your file is smaller than the blocksize, you'll have to reconstruct the file somewhere in order to get your datafile. – OneCricketeer Dec 31 '15 at 21:41

1 Answers1

4

You could read data into byte[], and then pass it to Store using VolumeFactory. However there is no public API for that yet. It is on my TODO list.

Jan Kotek
  • 1,084
  • 7
  • 4
  • Thanks Jan, I was able to access mapdb over hdfs with just few hours of coding. I am not reading files in byte array as it can cause OOM, rather I am reading files from hdfs. – Akash Mahajan Jan 04 '16 at 19:38
  • 1
    @Akash_Mahajan How did you actually solve this issue? Are you also able to store changes back to Hdfs? – KIC Mar 06 '17 at 18:59
  • @Jan Kotek Can you explain how to use MapDB API to read byte[] etc ? – Vimal Jain Jun 13 '18 at 08:22