Stack Overflow
Stack Overflow

Questions tagged [hdf]

Hierarchical Data Format (HDF, HDF4, or HDF5) is a set of file formats and libraries designed to store and organize large amounts of numerical data.

Hierarchical Data Format (HDF, HDF4, or HDF5) is a set of file formats and libraries designed to store and organize large amounts of numerical data.

Originally developed at the National Center for Supercomputing Applications, it is supported by the non-profit HDF Group, whose mission is to ensure continued development of HDF5 technologies, and the continued accessibility of data stored in HDF.

In keeping with this goal, the HDF format, libraries and associated tools are available under a liberal, BSD-like license for general use. HDF is supported by many commercial and non-commercial software platforms, including Java, MATLAB/Scilab, Octave, IDL, Python, and R. The freely available HDF distribution consists of the library, command-line utilities, test suite source, Java interface, and the Java-based HDF Viewer (HDFView).

There are two major versions of HDF; HDF4 and HDF5, which differ significantly in design and API.

Wikipedia: http://en.wikipedia.org/wiki/Hierarchical_Data_Format

344 questions
2
votes
1 answer

How to read HDF5 dataset with HDF.Pinvoke?

I started to use this https://github.com/HDFGroup/HDF.PInvoke to work with HDF5 files. But documentation is very poor. I want to read dataset. I use this function: H5D.read(...). The last parameter of this function is buf which type is…
Aleksandr Iurkin
  • 66
  • 8
2
votes
0 answers

Java fails to map a native library

I am having an issue where Java is trying to load an .so object but cannot map it, so it crashes. How can I debug in a situation like this? Here is the command (trying to run the HDFView utility,…
Zach Boyd
  • 419
  • 1
  • 5
  • 23
2
votes
0 answers

HDFview on Ubuntu 18.04

After updating ubuntu to the newest version 18.04 I have problem with HDFview. I can open hdfview but I can't load any file. I terminal I have this warning: [warning] /usr/bin/hdfview: No java runtime was found I quote some of info which I find:…
Mateusz
  • 21
  • 3
2
votes
1 answer

Strange layout of the HDF tables from pandas.HDFStore

When I output a pandas.DataFrame as a table in HDFStore: import pandas as pd df=pd.DataFrame({'A': [1, 2], 'B': [3, 4]}, index=range(2)) with pd.HDFStore("test.hdf5") as store: store.put("test", df, format="table") I get the following layout…
CharlesB
  • 86,532
  • 28
  • 194
  • 218
2
votes
1 answer

Model weights not showing properly in HDFView

I am saving model weight in keras using model.save_weights() which save weights in h5 extension. I seeing h5 file in HDFView2.9. My model summary looks like following: Layer (type) Output Shape Param # …
Hitesh
  • 1,285
  • 6
  • 20
  • 36
2
votes
1 answer

In Kafka, cannot create new consumer using kafka-consumer-groups.sh

I set up a Kafka standalone node(HDF ver 3.1.1, Kafka ver 1.0.0) To load data from kafka, one can create consumer group using kafka-consumer-groups.sh The command is like the following: kafka-consumer-groups.sh --bootstrap-server…
Lee. YunSu
  • 416
  • 6
  • 21
2
votes
2 answers

Python large dataset feature engineering workflow using dask hdf/parquet

There is already a nice question about it in SO but the best answer is now 5years old, So I think there should be better option(s) in 2018. I am currently looking for a feature engineering pipeline for larger than memory dataset (using suitable…
Florian Mutel
  • 1,044
  • 1
  • 6
  • 13
2
votes
0 answers

Python: Store HDF works first time, but no second

I have a dataframe which I store in a HDF file with the following command: # store data in HDF5 format store = pd.HDFStore('data.h5') store.put('data',data) store.close() It works all fine. In another code, I load the dataframe again, which also…
user27074
  • 627
  • 1
  • 6
  • 20
2
votes
0 answers

Load H5 file and store same data as new H5 file in Python: File size increases

I have an H5 file, named file.h5, which stores an infrared image. This file is of size 282KB: $ ls -l -sh file.h5 688 -rw-r--r-- 1 user staff 282K Feb 2 00:25 file.h5 First, I load the file in python using the library h5py. >> import h5py >>…
lucasrodesg
  • 638
  • 1
  • 6
  • 22
2
votes
1 answer

Obtain the HDF5 file format version of a file

Given a certain HDF5 file (HDF5 format), how can I know the version of the file format that has been used for its creation? Context. It is said in the documentation that when you create/write a file the library the following applies for backward…
pablo_worker
  • 1,042
  • 9
  • 26
2
votes
1 answer

Reading array of floats from HDF5 file in Fortran

I can create a HDF5 file in Python using h5py, e.g. import numpy as np import h5py outfile = np.zeros((5,2)) for i in range(5): outfile[i] = i print(outfile) print(outfile.dtype) f = h5py.File('TF.hdf5',…
user1887919
  • 829
  • 2
  • 9
  • 24
2
votes
0 answers

Modify, remove or add data to hdf5 files with R

I have a hdf5 file with several attributes, each attribute is either a matrix or an array. I would like to modify some of the attributes by adding or removing a column to the matrix. How can this be done? I have tried with the rhdf5 library in R but…
Herman Toothrot
  • 1,463
  • 3
  • 23
  • 53
2
votes
2 answers

Creating a 3D array in C++ using passed in parameters

I have a function that takes in a void* buffer parameter. This function (which is provided by HDF here. From my understanding, it reads info from a dataset into the buffer. I have this working, but only if I create a 3d int array using constant…
Roka545
  • 3,404
  • 20
  • 62
  • 106
2
votes
1 answer

Saving Pandas DataFrames as a HDF5 store, various errors

Just want to archive some Pandas DataFrames in a HDF5 store (.h5 file). Below is the code I'm using. # Fake data over N runs Data_N = [] for n in range(5): Data_N.append(np.random.randn(5000,15,125)) # Create HDFStore object store =…
leka0024
  • 31
  • 5
2
votes
1 answer

wide vs long format when saving data in pandas hdf5

pandas data frame are in general represented in long ( a lot of rows) or wide (a lot of columns) format. I'm wondering which format is faster to read and occupies less memory when saved as hdf file (df.to_hdf). Is there a general rule or some…
Donbeo
  • 17,067
  • 37
  • 114
  • 188
1 2 3
…
22 23