Stack Overflow
Stack Overflow

Questions tagged [pytables]

A Python library for working with extremely large hierarchical (HDF5) datasets.

PyTables is a package for managing hierarchical (HDF5) datasets and designed to efficiently and easily cope with extremely large amounts of data. PyTables is available as a free download.

PyTables is built on top of the HDF5 library, using the Python language and the NumPy package. It features an object-oriented interface that, combined with C extensions for the performance-critical parts of the code (generated using Cython), makes it a fast, yet extremely easy to use tool for interactively browse, process and search very large amounts of data.

Links to get started:
- Documentation
- Tutorials
- Library Reference
- Downloads

617 questions
0
votes
1 answer

CArray and EArray compatibility outside of PyTables

I use PyTables and often make use of CArray and EArray classes for their support of compression. I'm working on a project that may need compatibility outside of Python (in Java, most likely), however. Does that mean I can't use CArray and EArray? Or…
Jason S
  • 184,598
  • 164
  • 608
  • 970
0
votes
1 answer

Using pytables with python3 fails

importing pytables (3.1.1) in python 3.4.1 fails for me, complaining about a failed cPickle import i try to use potables (3.1.1) with python (3.4.1) In [1]: import…
user1316256
  • 1
  • 1
0
votes
2 answers

How to avoid high memory usage in pytables?

I am reading in a chunk of data from a pytables.Table (version 3.1.1) using the read_where method from a big hdf5 file. The resulting numpy array has about 420 MB, however the memory consumption of my python process has gone up by 1.6GB during the…
Ben K.
  • 1,160
  • 6
  • 20
0
votes
1 answer

What is the best way to get the number of elements in a PyTables row iterator?

My current approach is: rowiter = atable.where(condition) rowiter_length = max([i for i, row in enumerate(rowiter)]) Is there a way to get the length of rowiter without looping through the entire iterator?
user12345678
  • 429
  • 1
  • 4
  • 10
0
votes
2 answers

pytables, add rows in nested field

Starting from a simple structure like this one: from tables import * class subTable(IsDescription): subCol1= Int64Col(pos=0) subCol2= StringCol(itemsize=32, pos=1) subCol3= Int64Col(pos=2) class mainTable(IsDescription): column1=…
codeKiller
  • 5,493
  • 17
  • 60
  • 115
0
votes
1 answer

Pytables reading CArray very slow

I created a chunked array by: import tables FILTERS = tables.Filters(complib='lzo', complevel=1) h5file = tables.openFile('file.h5', mode='w', filters=FILTERS) x = h5file.createCArray(h5file.root,'chunk_array',tables.Float64Atom(), …
user2863620
  • 635
  • 3
  • 10
  • 17
0
votes
1 answer

How can I read HDF5 Time64 columns in IDL?

In Python (using pytables), it is easy to create HDF5 tables with rows containing timestamps (column datatype Time64, see http://pytables.github.io/usersguide/datatypes.html). Is it possible to read in tables containing columns with type Time64 in…
bdoering
  • 169
  • 7
0
votes
1 answer

ViTables Plotting Plugin

Hdf5View supports a very basic plotting feature. Although this feature is really simple, this has proven to be very useful to eyeball some data at first glance. Does ViTables have a similar feature? If not does anybody know a good Plugin for…
SmCaterpillar
  • 6,683
  • 7
  • 42
  • 70
0
votes
1 answer

correct query in PyTables

I am using Python 2.7, and trying to perform a PyTables query: #Here the condition selectedIndex = [1,6,7,9] condition = 'IndexColumn in selectedIndex' #here the query for x1 in tab.where(condition,selectedIndex): ... ... ... And I am…
codeKiller
  • 5,493
  • 17
  • 60
  • 115
0
votes
0 answers

more efficient solution for QTableWidget write

I am reading a PyTable, with 1320000rows x 16cols The idea is to read the table and to write its content into a QTableWidget. The way I am doing it makes the GUI collapse. I would like a clue about how to do it in an efficient way. Here it is my…
codeKiller
  • 5,493
  • 17
  • 60
  • 115
0
votes
2 answers

PyTables: Select Rows from Table and apply Numexpr

Is it possible to select rows from a table in PyTables and apply a Numexpr-like expression to the output? For example, if I have the table import tables as tb class Event(tb.IsDescription): x = tb.Float32Col() y = tb.Float32Col() z =…
xvtk
  • 1,030
  • 2
  • 10
  • 18
0
votes
1 answer

Searching through lots of logs (preferably in Python)

The problem: find log lines from time between say 2 months ago and 1 month ago and those log lines have to contain several specified words (not even regexes are necessary, though it would be nice to have them). The catch: there's 20T+ of logs…
LetMeSOThat4U
  • 6,470
  • 10
  • 53
  • 93
0
votes
0 answers

Anaconda distribution of pytables is broken

I just switched to the anaconda distribution of python, and I'm having trouble with a few packages. One example is pytables: ~$ ipython Python 2.7.6 |Anaconda 1.8.0 (64-bit)| (default, Jan 17 2014, 10:13:17) Type "copyright", "credits" or "license"…
arwright3
  • 381
  • 2
  • 3
  • 13
0
votes
0 answers

Pandas import excel export HDF5

Working with pandas and PyTables. Begin by importing a table from excel containing columns of integers and floats, as well as other columns containing strings and even tuples. There are a limited number of options on the excel import and…
riddley_w
  • 249
  • 1
  • 4
  • 7
0
votes
1 answer

how to set python environment variables

I've slowly been working my way through all of the dependencies required to get PyTables installed. That included updating Numpy, install numexpr and cython and the hdf5 libraries. I am now getting an error that states it cannot find a local HDF5…
marcp
  • 1,179
  • 2
  • 15
  • 36
1 2 3
…
41 42