compressing array with pytables

Question

I am trying to compress my array like this

import numpy as np
import tables
from contextlib import closing

FILTERS = tables.Filters(complib='zlib', complevel=5)

data = np.zeros(10**7)

with closing(tables.open_file('compressed', mode='w', filters=FILTERS)) as hdf:
    hdf.create_array('/', 'array', obj=data)

with closing(tables.open_file('uncompressed', mode='w')) as hdf:
    hdf.create_array('/', 'array', obj=data)

but it does not work at all

-rw-rw-r-- 1 user user 80002360 2013-11-21 15:27 compressed
-rw-rw-r-- 1 user user 80002304 2013-11-21 15:28 uncompressed

Am I doing something wrong here?

Answer 1

Arrays themselves cannot be compressed. Compression requires chunking and so you must either use chunked arrays (CArrays) or extendable arrays (EArray) instead. This is probably a 1 character change since you just want to call the create_carray() method instead of the create_array() method.

import numpy as np
import tables
from contextlib import closing

FILTERS = tables.Filters(complib='zlib', complevel=5)

data = np.zeros(10**7)

with closing(tables.open_file('compressed', mode='w', filters=FILTERS)) as hdf:
    hdf.create_carray('/', 'array', obj=data)

with closing(tables.open_file('uncompressed', mode='w')) as hdf:
    hdf.create_array('/', 'array', obj=data)