Stack Overflow
Stack Overflow

Questions tagged [zstd]

Use with questions related to zstd library. This library is an open-source implementation of Zstandard, a lossless data compression algorithm developed by Yann Collet at Facebook.

91 questions
2
votes
1 answer

ZStd. Heap corruption after decompression

In most cases it work good. For example, I compress 3D mesh. Almost all models compress\decompress good. But 2 or 3 models can give error when program try decompress it. It decompress good, but error when I free memory // how I read my files …
Тёма Басов
  • 33
  • 7
2
votes
1 answer

Decompressing tar.zst file-- why am I getting a premature end error?

I have been trying to download this dataset through my Mac terminal. I know it's huge! https://zenodo.org/record/3606810 I have the tar.zst file, and when I try to decompress (using zstd -d pol_0616-1119_labeled.tar.zst), it throws me this…
Ellen Jean Chapin
  • 21
  • 1
  • 1
  • 2
2
votes
2 answers

LZ4 and Zstd for Java

Is there any Best Java Compression Library available for LZ4 and ZStd. I have tried with apache commons ( which is zstd-jni implementation) String fileURL = TestFileUtil.getFileURL(TestFileCategory.SMALL); String…
Hulk_avenger
  • 21
  • 1
  • 2
2
votes
0 answers

Error: "Decoding error (36) : Dictionary mismatch" using ZSTD decode on command line to decode a .ZST file

I am trying to decode a bunch of .ZST files which I do not know what the original file was so I can access them, however all of the .ZSTs return the exact same error and do not get decompressed. The error is Decoding error (36) : Dictionary…
jhm2k
  • 53
  • 9
2
votes
5 answers

How to compress a C++ string using Zstd?

I'm very new to C++ and I wanted to compress a std:string object via Zstd compression library, but so far I couldn't find a C++ sample code for this purpose by googling it. I've found example C code but it seems that they are working with C style…
Hassan Abedi
  • 790
  • 1
  • 7
  • 22
1
vote
0 answers

What exactly does each ZSTD compression level do?

The ZSTD compression library by Facebook offers over 19 compression levels, but sometimes some seem to have more effect on overall compression efficiency and compression time than others. What exactly is the difference between, for e.g., level 5 and…
Wallace Long
  • 21
  • 2
1
vote
0 answers

zstd: can't stat web.log.* : No such file or directory -- ignored

I can't seem to find help on using zstdgrep correctly. I thought it works like ordinary grep, but I am not getting what I want. I have several subfolders that have files in the format of webserver.log.YYYY-MM-DD.zst. These are archives of logs over…
Tomáš Zato
  • 50,171
  • 52
  • 268
  • 778
1
vote
0 answers

"Decompressing large Zstandard file in Rust results in 'Frame requires too much memory for decoding' error"

I'm using the zstd-rs crate to decompress a large Zstandard file (~30GB) in Rust. My code opens the file, reads it into a buffer, and then uses the copy_decode function from the zstd::stream module to decompress the data and write it to an output…
weiserhase
  • 11
  • 3
1
vote
1 answer

ZSTD compression with tar and --ultra 22 on Mac OSX and homebrew (with progress bar)

I want to be able to compress a file or folder, and see the progress of said input in the terminal. This is my current progress so far, but I have two issues with it: This should be possible without an interim step of creating the tar file The…
OpenGears
  • 102
  • 7
1
vote
0 answers

Unknown frame descriptor while reading zst file on hive?

I have a file a.csv.zst and i ve created textfile table. When i select on hive i see first 100 rows without an error but when i select count(*) it gives an error unknown frame descriptor I can decompress my file on linux with zstd -dc a.csv.zstd How…
CompEng
  • 7,161
  • 16
  • 68
  • 122
1
vote
1 answer

"No enum constant org.apache.orc.CompressionKind.ZSTD" When Insert Data to ORC Compress ZSTD Table

I have created a table in hive 3.1.3 as below; Create external table test_tez_orc_zstd ( Id bigint )stored as orc Tblproperties(orc.compress=zstd) Location '...' It is created, and then I wanted to insert one row; Insert into…
CompEng
  • 7,161
  • 16
  • 68
  • 122
1
vote
1 answer

How to add zstd and lz4 algorithm to tar on MacOS?

So my tar version on MacOS doesn't seem to support the zstd or lz4 algorithms, and I am unsure how to update tar or add support for it. My Ubuntu's tar version seems to have support for zstd.
Dr. Light
  • 126
  • 7
1
vote
1 answer

Zstandart(zstd) - checking return

Can you please explain how the "ZSTD_isError" function works? I check the return code from ZSTD_isError to be zero: ZSTD_cParameter cParam; ZSTD_bounds ZSTD_bounds_ = ZSTD_cParam_getBounds(cParam); status_error =…
Serg_
  • 67
  • 5
1
vote
1 answer

ZSTD equivalent of ZLIB deflateInit2 method

In a c++ project I have a function using deflateInit2 method of ZLIB. ZEXTERN int ZEXPORT deflateInit2 OF((z_streamp strm, int level, int method, …
Vahag Chakhoyan
  • 873
  • 1
  • 10
  • 21
1
vote
1 answer

cannot enable redis-zstd in php:8.0.2-fpm-buster

In a container based on "php:8.0.2-fpm-buster" image, I run pecl install zstd echo "extension=zstd.so" > /path/to/php.ini then run pecl install redis enable zstd compression support? [no] : yes The output is checking for libzstd files in default…
Steve
  • 41
  • 1
  • 7
1 2
3
4 5 6 7