1

I am new with using NVIDIA DIGITS. My train dataset has following structure and its format is .hdf5 .

crops       Dataset {27482, 3, 128, 192}
labels      Dataset {27482, 12}
mean        Dataset {3, 128, 192}
pids        Dataset {27482}

I know how to feed model with simpler formats like .txt or .jpg. My question is how can i feed my model with .hdf5 format in NVIDIA DIGITS

MIRMIX
  • 1,052
  • 2
  • 14
  • 40

1 Answers1

1

HDF5 datasets are only used for image classification datasets in DIGITS, and even then the support isn't very full-featured.

Why?

Caffe doesn't support HDF5 nearly as well as it supports LMDBs:

  • For large datasets, you have to break them up into separate files (see here)
  • Data is not prefetched - the whole dataset is read into memory at once (see here)
  • Data transformations are not supported with the HDF5Data layer (see here)

Since DIGITS is primarily Caffe-based for now, our main dataset format is LMDB. If/when we support more backend frameworks, we may decide to standardize on a more generic format like HDF5 or zipfiles.

Luke Yeager
  • 1,400
  • 1
  • 17
  • 30