Stack Overflow
Stack Overflow

Questions tagged [h5py]

h5py is a NumPy-compatible Python module for handling The Hierarchical Data Format (HDF5) files.

h5py is a NumPy-compatible Python module for handling The Hierarchical Data Format (HDF5) files.

Main features

  • Free (BSD licensed)
  • limited dependencies (Python, NumPy, HDF5 libs.)
  • includes both a low level c-like HDF5 interface and a high level Python/NumPy style interface
  • directly interact with datasets using NumPy metaphors, such as slicing
  • datatypes specified using standard NumPy dtype objects

Some links to get started

1301 questions
0
votes
2 answers

Excluding a set of indices from an array whilst keeping its original order

I have an HDF5 dataset which I read as a numpy array: my_file = h5py.File(h5filename, 'r') file_image = my_file['/image'] and a list of indices called in. I want to split the image dataset into two separate np.arrays: one containing images…
ga97rasl
  • 307
  • 2
  • 7
  • 15
0
votes
0 answers

How to structure dask to stream to random forrest classifier

I have a number of h5py datasets in 1 file where the class label is the dataset name and the shape is (20000,250000) of float64 compressed using gzip How would the community suggest I use dask to enable random forrest training without needing to…
mobcdi
  • 1,532
  • 2
  • 28
  • 49
0
votes
1 answer

How to create a specific type of object for type .hdf5?

My question is about creating an object type or document for .hdf5 files. The object will have three attributes, an id, a user_id and a boolean array of size 64. I have to create them about 10000000 (Ten millions) many. Imagine mongodb, I have to…
A Ef
  • 13
  • 2
  • 8
0
votes
0 answers

Hdf5 file over a thousand times bigger than the sum of its parts

I have an HDF5 file that shows up as 23G on the filesystem. This seemed too big for the problem I am currently working on, so I decided to investigate. The file contains 70 datasets of roughly 100,000 instances each (datatype is int8, compressed…
levesque
  • 8,756
  • 10
  • 36
  • 44
0
votes
1 answer

Installing h5py on 64 bit Windows

I'm trying to install h5py onto a 64 Windows 7 system. In the past I've had success using the wheel files at http://www.lfd.uci.edu/~gohlke/pythonlibs/ Unfortunately the 64 bit version does not seem to be available. I get the following…
JohnSantaFe
  • 145
  • 1
  • 3
  • 11
0
votes
0 answers

Forcing string type in compound data type

I have an hdf5 file containing a group with an attribute of the following format (h5dump): ATTRIBUTE "PrioInfo" { DATATYPE H5T_COMPOUND { H5T_STRING { STRSIZE 128; STRPAD H5T_STR_NULLTERM; CSET…
user1479670
  • 1,145
  • 3
  • 10
  • 22
0
votes
1 answer

Using PyTables to index a 500 GB HDF5 file

I would like to dump a keyed 500GB-800GB table into HDF5, and then then retrieve rows matching specific keys. For an HDF5 file, items like all the data access uses an integer "row" number, so seems like I would have to implement a 'key to row number…
ShanZhengYang
  • 16,511
  • 49
  • 132
  • 234
0
votes
0 answers

Cannot convert TIFF to an array

I have two datasets of .tif files. For data_1, I can convert the .tif images to arrays by using scipy.imread() and np.array. But for the other dataset data_2, when I apply the same functions, the .tif files are not converted to float arrays. import…
esw12345
  • 91
  • 1
  • 1
  • 7
0
votes
0 answers

How to upload really large datasets to an hdf5 file?

Traceback (most recent call last): File "populate_h5.py", line 116, in dset_X[n_images:n_images+1,:,:,:]=hc File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (-------src-dir--------/h5py/_objects.c:2582) File…
BigBoy1337
  • 4,735
  • 16
  • 70
  • 138
0
votes
1 answer

Default value of kwarg 'compression' in create_dataset() of h5py

what is the default value of the compression keyword argument of the function create_dataset in h5py, i.e. what value do I have to pass to get no compression? I'm asking because I would like to have a flag which turns the compression on/off and I'd…
obachtos
  • 977
  • 1
  • 12
  • 30
0
votes
2 answers

Opening a mat file using h5py and convert data into a numpy matrix

I have a mat file which contains 2 different cells containing matrices of different size. I need to convert that data into a numpy array using h5py for an experiment (I'm new in h5py. I thought it was as easy as it is explained here Reading the…
Diego Alejandro Gómez Pardo
  • 391
  • 6
  • 18
0
votes
1 answer

Error installing h5py using pip

Error installing h5py using pip Environment:- spark service IBM bluemix !pip install --user h5py fails with error gcc failed. I even tried to download the package and then run !python setup.py install
charles gomes
  • 2,145
  • 10
  • 15
0
votes
0 answers

How to add data to H5py data?

I have created dataset with 4 names. and by default has only one value each. How do I add another vaue to each group'lstm_X' h5f = h5py.File('data.h5', 'w') h5f.create_dataset('lstm_X', data=0) h5f.create_dataset('lstm_Y',…
Arsenal Fanatic
  • 3,663
  • 6
  • 38
  • 53
0
votes
0 answers

Transpose a matrix created by h5py

I have a h5py data that looks like this. import h5py f = h5py.File('X.h5', 'w') f['X'] = np.array([[0, 0], [0, 1], [1, 0], [1, 1]]) >>> f['X'][...] array([[0, 0], [0, 1], [1, 0], [1, 1]]) I want transpose of…
Kavan
  • 331
  • 1
  • 4
  • 13
0
votes
1 answer

suppress numpy array creation protocol for numpy arrays of objects

I am trying to build a library which reads complex HDF5 data files in python. I am running into a problem where, an HDF5 Dataset somehow implements the default array protocol (sometimes), such that when a numpy array is created from it, it casts to…
aestrivex
  • 5,170
  • 2
  • 27
  • 44
1 2 3
…
86 87