Stack Overflow
Stack Overflow

Questions tagged [hdf]

Hierarchical Data Format (HDF, HDF4, or HDF5) is a set of file formats and libraries designed to store and organize large amounts of numerical data.

Hierarchical Data Format (HDF, HDF4, or HDF5) is a set of file formats and libraries designed to store and organize large amounts of numerical data.

Originally developed at the National Center for Supercomputing Applications, it is supported by the non-profit HDF Group, whose mission is to ensure continued development of HDF5 technologies, and the continued accessibility of data stored in HDF.

In keeping with this goal, the HDF format, libraries and associated tools are available under a liberal, BSD-like license for general use. HDF is supported by many commercial and non-commercial software platforms, including Java, MATLAB/Scilab, Octave, IDL, Python, and R. The freely available HDF distribution consists of the library, command-line utilities, test suite source, Java interface, and the Java-based HDF Viewer (HDFView).

There are two major versions of HDF; HDF4 and HDF5, which differ significantly in design and API.

Wikipedia: http://en.wikipedia.org/wiki/Hierarchical_Data_Format

344 questions
3
votes
1 answer

Columns index has to be unique for fixed format error in pandas time series

I got ValueError: Columns index has to be unique for fixed format when I tried to save a dataframe which I formed by combining multiple time series dataframe. This is the sample of what I have…
Eka
  • 14,170
  • 38
  • 128
  • 212
3
votes
3 answers

apache nifi - how to execute process group one after the other

I have created 3 process groups. I want second process group to be executed after first process group is completed, and third process group to be executed after second process group is completed. Any solution on this please??
priyanka
  • 287
  • 2
  • 5
  • 11
3
votes
1 answer

How to write stl::string to HDF5 file in c++

I am using HDF5 CPP library on windows platform. Basically I want write a compound datatype to H5 file which contain std::string. This code is not giving any error but while writing to H5 file it writes garbage value... Guys any hint is…
Shamkumar Rajput
  • 166
  • 1
  • 10
3
votes
2 answers

On disk indexing of Pandas multiindexed HDFStore

In order to improve performance and reduce memory footprint, I am trying to read a multi-indexed HDFStore created in Pandas. The original Store is quite large, but the problem can be reproduced with a similar but smaller example. df =…
Suraj
  • 53
  • 5
3
votes
2 answers

Pandas: Large dataframes in the same HDF?

I have several different data frames that are related (and there is ids to join them if needed). However, I don't always need them at the same time. Since they are quite large, does it make sense to store them in separate HDF stores? Or is the cost…
FooBar
  • 15,724
  • 19
  • 82
  • 171
3
votes
0 answers

Convert HDF5 file to JPG image

Is there an easy way to convert a HDF5 file to it's pictorial representation? I'm using HDFView currently to open an HDF5 file and view it as an image, is there a way I can do a conversion in python that mimics this and writes out a picture file?
Syntactic Fructose
  • 18,936
  • 23
  • 91
  • 177
3
votes
1 answer

Symbol not found - linking to hdf library

I am trying to use the hdf5-format to store data. Problem is, that I fail to link against the library. I have the following code #include int main(void){ H5::H5File file("test_MatrixRoundTrip_Double.h5", H5F_ACC_TRUNC); } and compile…
physicsGuy
  • 3,437
  • 3
  • 27
  • 35
3
votes
1 answer

How to put a line break in HDF5 attribute string in Matlab

I would like to know how to put a line break in below HDF5 attribute string. DescType_id = H5T.copy ('H5T_C_S1'); H5T.set_size (DescType_id, numel(description)); H5T.set_strpad(DescType_id,'H5T_STR_NULLTERM'); DescAttr_id = H5A.create (g2id,…
user4530590
  • 41
  • 2
3
votes
0 answers

Error Reading Pandas HDFStore (with PyTables)

I have a pandas HDFStore object (pandas uses PyTables to implement HDF5 if that is relevant) with a mydata DataFrame (store["mydata"] = mydata) and have some code that regularly appends data to the mydata DataFrame. When I try to access that…
Ty Pavicich
  • 1,050
  • 3
  • 9
  • 24
3
votes
1 answer

Most efficient way to get index of a table from HDF5

I have an HDF5 file containing pandas Series/DataFrame tables. I need to get (pandas) index of a table stored under a key in HDF, but not necessarily the whole table: I can think of two (effectively the same) methods of getting the index: import…
EKomarov
  • 61
  • 1
  • 5
3
votes
2 answers

Using std:: string in hdf5 creates unreadable output

I'm currently using hdf5 1.8.15 on Windows 7 64bit. The sourcecode of my software is saved in files using utf8 encoding. As soon as I call any hdf5 function supporting std:: string, the ouput gets cryptic But if I use const char* instead of…
user2445483
  • 41
  • 1
3
votes
1 answer

Load HDF5 in Excel?

Is there a way to load/import an HDF5 file (sometimes called H5, HD5, or Hierarchical Data Format) in to Excel? I do not want to export an intermediary CSV file, as ideally I would like to modify the data and in Excel and resave back to HDF5 (but…
MD004
  • 581
  • 1
  • 7
  • 19
3
votes
1 answer

matlab read h5 file produced with pandas

I have a csv file and I have transformed it in an h5 file with pandas: data = pd.read_csv('file.csv') data.to_hdf('file.h5', 'table') Now I would like to read it with matlab. How can I do that? I have tried data =…
Donbeo
  • 17,067
  • 37
  • 114
  • 188
3
votes
1 answer

Alternative to npz files?

I use arrays stored in Python .npz format. I have a lot of these files, which all share the same common structure: filename my_file_var1_var2_var3.npz contains the following items (all arrays are 32bit Floats): a 2D array (N=11, Ns=2000) a 2D array…
Jose
  • 2,089
  • 2
  • 23
  • 29
3
votes
2 answers

Creating reference to HDF dataset in H5py using astype

From the h5py docs, I see that I can cast a HDF dataset as another type using astype method for the datasets. This returns a contextmanager which performs the conversion on-the-fly. However, I would like to read in a dataset stored as uint16 and…
achennu
  • 1,538
  • 1
  • 13
  • 18
1 2
3
…
22 23