Stack Overflow
Stack Overflow

Questions tagged [h5py]

h5py is a NumPy-compatible Python module for handling The Hierarchical Data Format (HDF5) files.

h5py is a NumPy-compatible Python module for handling The Hierarchical Data Format (HDF5) files.

Main features

  • Free (BSD licensed)
  • limited dependencies (Python, NumPy, HDF5 libs.)
  • includes both a low level c-like HDF5 interface and a high level Python/NumPy style interface
  • directly interact with datasets using NumPy metaphors, such as slicing
  • datatypes specified using standard NumPy dtype objects

Some links to get started

1301 questions
0
votes
1 answer

Preventing program from existing during file write

I'm regularly writing a 5 GB NumPy ndarray to disc using H5py. If I close the file during this write, it gets corrupted. Is there any way for me to tell the program something along the lines of, "if somebody ends your program, wait until you finish…
Eric Gorlin
  • 319
  • 1
  • 8
0
votes
2 answers

Setting and reading dimscale in hdf5 files correctly in python

I am trying to attach dimension scales to datasets I want to store in hdf5 files with python, but get an error when I try to print the attributes after setting them. The relevant code snippet reads as follows: import h5py import numpy as np #…
Alf
  • 1,821
  • 3
  • 30
  • 48
0
votes
1 answer

How to feed a regular expression to a method that requires a string?

What I want to achieve is based on the voted answer for a previous question: Check if node exists in h5py Basically I want to replace: "/some/path" in h5File with something like: import re re.compile(r'/some/[pattern]+') in h5File
Hailiang Zhang
  • 17,604
  • 23
  • 71
  • 117
0
votes
1 answer

Python multiprocessing with arrays and multiple arguments

So I am trying to read in a bunch of very large data files and each one takes quite some time to load. I am trying to figure out how to load them in the quickest way and without running into memory problems. Once the data files are loaded into the…
Steven Walton
  • 505
  • 6
  • 20
0
votes
1 answer

Iterate over an HDF file with groups and subgroups with python itertools

I would like to parse an HDF file that has the following format HDFFile/ Group1/Subgroup1/DataNDArray ... /SubgroupN/DataNDArray ... GroupM/Subgroup1/DataNDArray ... /SubgroupN/DataNDArray I am…
Satrapes
  • 183
  • 1
  • 18
0
votes
1 answer

store image into hdf5 store?

I need to store the RGB image data (96*96*3) into the HDF5 datastore, similar to the example shownhere where train_data size: (42000, 1, 28, 28). My image data dimension is like (image_num, 3, 96, 96). Here is my code to store one image into the…
user824624
  • 7,077
  • 27
  • 106
  • 183
0
votes
1 answer

h5py will not overwrite when file has user block

I am new to HDF5 and I am trying to figure out how to open and overwrite an HDF5 file that has a user block in python 2.7 and using h5py version 2.1.3. Lets say I create a file in the following way: import numpy as np import h5py def…
veda905
  • 782
  • 2
  • 12
  • 32
0
votes
2 answers

h5py returning unexpected results in indexing

I'm attempting to fill an h5py dataset with a series of numpy arrays that I generate in sequence so my memory can handle it. The h5py array is initialised so that the first dimension can have any magnitude, f.create_dataset('x-data', (1, maxlen,…
user3297367
  • 79
  • 1
  • 1
  • 9
0
votes
1 answer

hdf5 warning with simple key check in h5py

When I open an h5 file file.h5 with h5py and check for a certain key: then this does not work: found = "data" in h5File.keys() # warning on this line if found: a = h5File["data"][...] and spits out the following warning: HDF5-DIAG: Error…
Gabriel
  • 8,990
  • 6
  • 57
  • 101
0
votes
1 answer

h5py Ignore external links when reading file

I am reading in a group from an hdf5 file, in which some of the group members are external links. I would like to be able to identify which group members are external links, and subsequently ignore them for further reading. Example code: f =…
Samuel
  • 157
  • 6
  • 22
0
votes
1 answer

Attribute error in 'file' and 'File' object in hdf5 file

I have created a hdf5 file using file = open() command. In this case, I can write and read the file. But it is giving me attribute error when I am trying file.keys(). The error is AttributeError: 'file' object has no attribute 'keys'. Then I have…
sm10
  • 9
  • 1
  • 5
0
votes
1 answer

In python, can I see if a file lives on an HD or an SSD?

I want to randomly access the elements of a large array (>7GB) that I load into Python as a either an HDF5 dataset (h5py.Dataset), or a memory-mapped array (numpy.memmap). If this file lives on an spinning-platter HD, these random accesses take…
SuperElectric
  • 17,548
  • 10
  • 52
  • 69
0
votes
1 answer

Reading gzipped data in Python

I have a *.tar.gz compressed file that I would like to read in with Python 2.7. The file contains multiple h5 formatted files as well as a few text files. I'm a novice with Python. Here is the code I'm trying to adapt: …
DJohnson
  • 148
  • 11
0
votes
1 answer

How can I install h5py with Portable Python v2.7?

I am doing some Python 2.7 development work on multiple computers, on some of which I do not have rights to install software. Thus, I am using Portable Python. One of the packages I need to use is h5py; however, since Portable Python installs…
hBy2Py
  • 1,707
  • 19
  • 29
0
votes
2 answers

Print out the data an hdf file contains (either to a file or as a chart)

If I'm given *.hdf file, how can I print out all the data it contains? >>> import h5py >>> f = h5py.File('my_file.hdf', 'r') >>> # What's next? All the questions here describe how to either create an hdf file or just read it without printing out…
user266003
1 2 3
…
86 87