Questions tagged [blosc]
5 questions
2
votes
1 answer
How to match all variables in xarray encoding (blosc, zarr compression)
The example of how to use zarr compression has the following code example see xarray doc:
In [42]: import zarr
In [43]: compressor = zarr.Blosc(cname="zstd", clevel=3, shuffle=2)
In [44]: ds.to_zarr("foo.zarr", encoding={"foo": {"compressor":…
marscher
- 800
- 1
- 5
- 22
1
vote
1 answer
blosc.MAX_BUFFERSIZE error while trying to guess if a dask dataframe is empty
I want to perform a test on the emptiness of a dask dataframe.
So I have this dask dataframe ddf, a local ray cluster, and dask configured to use ray as backend.
I've seen here that there is no empty property and that I have to perform the following…
guillaume latour
- 352
- 2
- 18
1
vote
0 answers
Anaconda 2020.07 with python3.8 lacks support for 'snappy' compressor in blosc?
I'm loading an hdf file wrote under pandas.to_hdf(...,complib="blosc:snappy") in python3.7 installed by anaconda
after I upgraded anaconda to py3.8, it shows
HDF5ExtError: HDF5 error back trace
File "C:\ci\hdf5_1545244154871\work\src\H5Dio.c", line…
aEgoist
- 31
- 3
1
vote
0 answers
how to store metadata with numpy array compressed with blosc
I am currently compressing a numpy array using blosc with the following:
view = memoryview(large_np_arr)
compressed = blosc.compress(view, typesize=8)
and decompressing like so:
decompressed = blosc.decompress(compressed_view)
decompressed_arr =…
morfys
- 2,195
- 3
- 28
- 35
0
votes
1 answer
Changing compression parameter in python-blosc2
I wanted to test out python-blosc2.
When trying do compress data with a user-defined Filter however, I stumbled across a for me unexplainable error.
import blosc2
import numpy as np
a = np.random.rand(1000, 1000)
blosc2.compress(a,…
Helmut
- 311
- 1
- 9