Stack Overflow
Stack Overflow

Questions tagged [hdf5]

The Hierarchical Data Format (HDF5) is a binary file format designed to store large amount of numerical data.

HDF5 refers to:

  • A binary file format designed to store efficiently large amount of numerical data
  • Libraries of function to create and manipulate these files

Main features

  • Free
  • Completely portable
  • Very mature
  • No limit on the number and size of the datasets
  • Flexible in the kind and structure of the data and meta-data
  • Complete library in C and Fortran well documented
  • A lot of wrappers and tools are available (Python, Matlab, Java, …)

Some links to get started

2598 questions
1
vote
2 answers

Visualize large data written by ILNumerics API using HDFView?

I write my application data as H5Datasets using ILNumerics api. With small size data (5 X 4098), I am able to visualize the data using HDFView(version 2.9). But for the data of size 30 x4098 and more, HDFView crashes silently as I try to look at…
Neelima
  • 243
  • 1
  • 8
1
vote
1 answer

How do I read/write to a subgroup withing a HDF5Store?

I am using the HDF5Store, to store some of my processed results, prior to analysis. Into the store I want to put 3 types of results, Raw results, that have not been processed at all, just read-in and merged from their original CSV formats Processed…
Frames Catherine White
  • 27,368
  • 21
  • 87
  • 137
1
vote
2 answers

Append all rows from one table to another using Pytables

Let's take for example the following: import tables import numpy as np # Two Example Tables hfile = tables.open_file('myfile.h5', 'a') data1 = np.ones((3, 2)) data1.dtype = [('a', float), ('b', float)] data2 = np.zeros((3, 2)) data2.dtype = [('a',…
jtorca
  • 1,531
  • 2
  • 17
  • 31
1
vote
1 answer

Simultaneously indexing fields and a slice using `h5py`

Given an H5Py file of the following type In [41]: d.shape Out[41]: (37450461,) In [42]: d.dtype Out[42]: dtype([('transaction', '
MRocklin
  • 55,641
  • 23
  • 163
  • 235
1
vote
1 answer

Compress only selected variables when saving to .mat

I have two variables data and meta, which I am saving in a compressed .mat file (version '-v7'). The data variable is usually 800mb uncompressed, while meta might not even be 1mb. I have lots of these .mat files and sometimes I just need to loop…
Oleg
  • 10,406
  • 3
  • 29
  • 57
1
vote
0 answers

Writing dataset of type H5T_ARRAY

I'm trying to write data in HDF5 using the C++ API. I work on Windows XP 64 bits with Visual Studio 2010. I use version 1.8.9. The target is set to X86 so I had to use the 32 bits version of HDF (to be honest, I'm very new to programming with…
user3738172
1
vote
0 answers

How to read efficiently many hdf5 objects in R?

I'm using the 'rhdf5' package for reading a large file (2GB) with about 5000 objects. I have to use this package since it appears to be the only one supporting bit64. The problem is the following, it is very time consuming to read all the objects…
Joe
  • 75
  • 3
1
vote
0 answers

viTables and HDFView Memory Errors when opening large CArray

Both viTables and HDFView return memory errors when opening a large carray generated from pytables code. It appears that the viewer attempts to read the entire carray into memory and fails. Is this actually the case or am I mising something? Open…
tnt
  • 3,411
  • 5
  • 24
  • 23
1
vote
1 answer

Writing a comma separated string attribute to HDF5 dataset C++

I am trying to write a comma separated string attribute to an HDF5 dataset. I create the data set using, dset = H5Dcreate(file, "dset1", H5T_NATIVE_DOUBLE, file_space, H5P_DEFAULT, plist, H5P_DEFAULT); The data is basically columnar, it has fields…
ganesh reddy
  • 1,842
  • 7
  • 23
  • 38
1
vote
1 answer

How to filter for specific objects in a HDF5 file

Learning the ILNumerics HDF5 API. I really like the option to setup a complex HDF5 file in one expression using C# object initializers. I created the following file: using (var f = new H5File("myFile.h5")) { f.Add(new H5Group("myTopNode") { …
user492238
  • 4,094
  • 1
  • 20
  • 26
1
vote
2 answers

What is the PyTables counterpart of a SQL query "SELECT col2 FROM table WHERE col1 IN (val1, val2, val3...)"?

I'm looking for the PyTables counterpart of a SQL query in the form of... SELECT col2 FROM table WHERE col1 IN (val1, val2, val3...) ...where the condition values are stored in a Python list/tuple or numpy array of arbitrary length. The search…
THM
  • 745
  • 2
  • 8
  • 14
1
vote
1 answer

reading QString from HDF5

I have a hdf5 file containing some numbers in an attribute like this: 1.0:2.0,2.0:3.0,3.0:4.0 (More precisely the exact attribute: ATTRIBUTE "azangles" { DATATYPE H5T_STRING { STRSIZE H5T_VARIABLE; STRPAD…
smaica
  • 723
  • 2
  • 11
  • 26
1
vote
0 answers

HDF5 - Problems with including into a bigger C++ Project

I've got problems including a script to open and read data from an existing HDF5-File in a larger C++ Project. I could include the libraries and headers so that there is no longer the error message: "undefined reference" when trying to link all the…
smaica
  • 723
  • 2
  • 11
  • 26
1
vote
2 answers

HDF5 C++ read the contents of an attribute

I try to read the contents of an Attribute. I use C++ and the HDF5 API. My script looks like this: #include #include #ifndef H5_NO_NAMESPACE #ifndef H5_NO_STD using std::cout; using std::endl; #endif // H5_NO_STD #endif #include…
smaica
  • 723
  • 2
  • 11
  • 26
1
vote
1 answer

HDF5 mounting files (with H5Fmount)

I try to mount different files that all have the same structure to a single new one (in C++). Therefore I create a new File "new.h5" and in there a new Group "/G". The existing two files ("file1.h5" and "file2.h5") have some Attributes in the…
smaica
  • 723
  • 2
  • 11
  • 26
1 2 3
…
99
100