32

I've just discovered the HDF5 format and I'm considering using it to store 3D data spread over a cluster of Java application servers. I have found out that there are several implementations available for Java, and would like to know the differences between them:

Most importantly, I would like to know:

  • How much of the native API is covered, any limitations that do not exist in the native API?

  • If there is support for "Parallel HDF5"?

  • Once my 3D data is loaded, do I get a "native call overhead" each time I access one element in a 3D array? That is, do the data actually gets turned into Java objects, or stay in "native/JNI memory"?

  • Is there any know stability problems with a particular implementation, since a crash in native code normally takes the whole JVM down?

troy.unrau
  • 1,142
  • 2
  • 12
  • 26
Sebastien Diot
  • 7,183
  • 6
  • 43
  • 85

3 Answers3

26

HDF Java follows a layered approach:

  • JHI5 - the low level JNI wrappers: very flexible, but also quite tedious to use.

  • Java HDF object package - a high-level interface based on JHI5.

  • HDFView - a Java-based viewer application based on the Java HDF object package.

JHDF5 provides a high-level interface building on the JHI5 layer which provides most of the functionality of HDF5 to Java. The API has a shallow learning curve and hides most of the house-keeping work from the developer. You can run the Java HDF object package (and HDFView) on the JHI5 interface that is part of JHDF5, so the two APIs can co-exist within one Java program.

Permafrost and Nujan seem far from being complete at this point and Permafrost hasn't seen a lot of activity recently, so they appear to be not the first choice at this point in time.

I think a good path for you is to have a look at both the Java HDF5 object package and JHDF5, decide which one of the two APIs fit your needs better and go with that one.

Disclaimer: I have worked on the JHDF5 interface, so I may be biased.

Bernd Rinn
  • 276
  • 3
  • 2
7

Just wanted to point out another option, jhdf.io it's a pure Java library for HDF5. Currently it is read only and doesn't cover the full HDF5 specification. However it can open and read lots of HDF5 files, and I hope to improve it over time. Being pure Java it is much easier to integrate into other Java projects than other options and avoids the issues associated with JNI.

Disclaimer: I am the author of the jhdf library.

James Mudd
  • 1,816
  • 1
  • 20
  • 25
0

Another API to handle HDF5 files in Java is HDFql. One of its main features is to be declarative. This makes handling HDF5 files (much) easier as you just tell HDFql what to do instead of telling how to do it when using one of the aforementioned APIs (all of them are imperative).

SOG
  • 876
  • 6
  • 10