Stack Overflow
Stack Overflow

Questions tagged [pytables]

A Python library for working with extremely large hierarchical (HDF5) datasets.

PyTables is a package for managing hierarchical (HDF5) datasets and designed to efficiently and easily cope with extremely large amounts of data. PyTables is available as a free download.

PyTables is built on top of the HDF5 library, using the Python language and the NumPy package. It features an object-oriented interface that, combined with C extensions for the performance-critical parts of the code (generated using Cython), makes it a fast, yet extremely easy to use tool for interactively browse, process and search very large amounts of data.

Links to get started:
- Documentation
- Tutorials
- Library Reference
- Downloads

617 questions
0
votes
1 answer

nested pytables

Suppose you are passing a dictionary to the pytable constructor: h5f.createTable('/','table',{'col1':Float64Col(pos=0),'col2':StringCol(16,pos=1)}) I have the following three beginner's questions related to nested pytables: 1) How do you use a…
Eugene Redekop
  • 81
  • 1
  • 5
0
votes
1 answer

Design strategy for managing and processing datasets with Pandas + PyTables

Being a regular user of numpy, I am falling in love with Pandas for its syntax and concepts, and started to read about PyTables, which seems very robust and performance-oriented. My main doubts are: If I were to use a home-made framework composed…
heltonbiker
  • 26,657
  • 28
  • 137
  • 252
0
votes
0 answers

Possible to Look Up Groups in Pytables through Dictionary?

I was wondering if its possible to store some kind of dictionary within pytables that could map an id to an instance of a group? Currently, I am trying to separate my data within groups; however, the lookup time to find these groups (through getNode…
jab
  • 5,673
  • 9
  • 53
  • 84
0
votes
1 answer

PyTables table.where equivalent in matlab

I'm trying to find something similar in MATLAB to PyTables' table.where that selects a subset of a dataset based on criteria (such as col1 = 4). So far, my searching has been completely fruitless. I can't believe such a useful feature wouldn't be…
uscere90
  • 543
  • 1
  • 5
  • 12
0
votes
1 answer

PyTables install fail at windows

I try to install pytable module. I use win7 x64, python 2.7.2 (x32), cython 0.16, visual studio 2008 (9.0). At cmd.exe 'run as administrator' write: 'python setup.py install'. But there are unresolved external symbols and build failed: * Found…
Max Tkachenko
  • 792
  • 1
  • 12
  • 30
0
votes
1 answer

Array order in pytables

With pytables's CArray, is there a way to specify the order in which the data is stored on disk (Fortran/C)? I am looking for something similar to ndarray's order parameter.
NPE
  • 486,780
  • 108
  • 951
  • 1,012
0
votes
1 answer

How to form multi-dimentional array from several 2-d arrays in NumPy / PyTables?

I have several 2-dimentional arrays under a hdf5 file, how can I merge them together into one multi-dimentional array in PyTables?i.e., the file now is array A, arra yB, array C, i want them to be array X with 1st dimension A, 2nd dimension B, 3rd…
l.z.lz
  • 393
  • 1
  • 4
  • 13
0
votes
1 answer

Adding NumPy array to existing HDF5 file in PyTables?

I have two sliced Numpy arrays stored in memory, say a and b, shape 1*480*640, how can I add them to an existing HDF5 file in PyTables? The existing HDF5 file has three arrays: n1,n2,n3, I want the new HDF5 file to have 5 arrays: n1,n2,n3,a,b . Thx!
l.z.lz
  • 393
  • 1
  • 4
  • 13
0
votes
1 answer

Query for bit based values

I have a problem how to get values from pytables. Values are bit based, but stored as integer number. One column in my table is Int32Column() with name 'Value'. In this column I will store integer values where every bit has different meaning. So, if…
Nikola Stevanovic
  • 251
  • 6
  • 18
-1
votes
1 answer

Although tables have already been installed on my pc, error appears every time when I run the code below

import pandas as pd import numpy as np import matplotlib.pyplot as…
Rehim Hesimov
  • 29
  • 5
-1
votes
1 answer

pytables and pandas string padding question

I've created a dataset using hdf5cpp library with a fixed size string (requirement). However when loading with pytables or pandas the strings are always represented like: b'test\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff The string value of 'test' with…
Moet
  • 1
-1
votes
1 answer

python multiprocessing (using pytable) misses some results from the queue in the final output

Before I state my question, let me put my constraint - I can't post the code as it is related to my job and they don't allow it. So this is just a survey query to see if somebody has seen similar issues. I have a python multiprocessing set up where…
manav
  • 217
  • 1
  • 2
  • 11
-1
votes
1 answer

Pytables vs Virtual Memory

Some code I use creates matrices larger than the RAM. My first thought was to go for pytables by modifying the current matrices to use pytables. But it will be a pain to do the coding (the code I use is opensource and doing the modifications is a…
eddys
  • 1,147
  • 1
  • 8
  • 14
-1
votes
1 answer

How to query a PyTables frame_table saved via a Pandas Dataframe?

I have the following pandas dataframe: import pandas as pd df = pd.read_table('fname.dat') So, I create/ open an existing HDFStore file: store = pd.HDFStore('store.h5') To index a subset of columns, I simply use store.append('key_name', df,…
ShanZhengYang
  • 16,511
  • 49
  • 132
  • 234
-1
votes
1 answer

UnicodeDecode error when reading special characters from HDFStore with Pandas

I need to store a lot of messages in HDFStore, some of them contain emoticons or special characters like éěščřžýáí. Everything seems to work ok until I try to load it, then it crashes with error below. Here is example code that ends up with the…
Filip Dušek
  • 37
  • 1
  • 3
1 2 3
…
41
42