Stack Overflow
Stack Overflow

Questions tagged [lz4]

LZ4 is a lossless data compression algorithm that is focused on compression and decompression speed

LZ4 is a lossless data compression algorithm that is focused on compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes.

Use this tag for questions that have issues with the implementations in one of the native languages or binding like c, c++, java, python.

LZ4 Repository

LZ4 Homepage

Adapted From Wikipedia

166 questions
2
votes
1 answer

Decoding the LZ4 output from iOS's compression_encode_buffer

I've got an iOS app compressing a bunch of small chunks of data. I use compression_encode_buffer running in LZ4 mode to do it so that it is fast enough for my needs. Later, I take the file[s] I made and decode them on a non-Apple device. …
ArtHare
  • 1,798
  • 20
  • 22
2
votes
0 answers

lz4 compression algorithm explaination

Can some one explain me the lz4 algorithm with an example byte array and how that is compressed? I am unable to understand how the compression algorithm is working and how it is so fast and still so promising about the compression ratio.
Sayantan Ghosh
  • 998
  • 2
  • 9
  • 29
2
votes
1 answer

How to use LZ4 compression functions in kernel modules?

I am using kernel 3.16 and trying to use LZ4 to compress data in memory. I checked kernel source tree, found the compression source file /lib/lz4.c and I used the following function: int lz4_compress(const unsigned char *src, size_t src_len, …
user3457702
  • 31
  • 1
  • 4
2
votes
2 answers

How to decompress less than original size with Lz4 library?

I'm using LZ4 library and when decompressing data with: int LZ4_decompress_fast_continue (void* LZ4_streamDecode, const char* source, char* dest, int originalSize); I need only first n bytes of the originally encoded N bytes, where n < N. So in…
mambo_sun
  • 509
  • 4
  • 13
2
votes
1 answer

Explain lz4 double buffer example

In lz4 examples, there is one named doublebuffer "https://github.com/Cyan4973/lz4/blob/master/examples/blockStreaming_doubleBuffer.c". This uses a char inpBuf[2][BLOCK_BYTES] during a read-compress loop and uses inpBuf[0][], inpBuf[1][] alternately.…
sivann
  • 2,083
  • 4
  • 29
  • 44
2
votes
4 answers

Compressing Double Values using C

I have a data set of double values (1024 x 1024 matrix). I am thinking to use lz4compression algorithm to compress this data. After compressing, I have to send this data to a server and then doing some processing at the server side and get it back…
Hamid
  • 137
  • 1
  • 11
1
vote
1 answer

How to add zstd and lz4 algorithm to tar on MacOS?

So my tar version on MacOS doesn't seem to support the zstd or lz4 algorithms, and I am unsure how to update tar or add support for it. My Ubuntu's tar version seems to have support for zstd.
Dr. Light
  • 126
  • 7
1
vote
0 answers

Ubuntu 18.04 LTS, ROS, Failed to load Python extention for LZ4 support. LZ4 compression will not be avilable

I'v been writing a code to extract images from bag file I don't get any error messages but this Failed to load Python extension for LZ4 support. LZ4 compression will not be available I use conda environment and my python version is 3.6.13 ros-roslib…
JeromeAtanasio
  • 11
  • 2
1
vote
0 answers

Python: Getting uncompressed file size of LZ4 file

I have compressed a large (>10 GB) file using the python lz4 package. I would like to later retrieve the uncompressed file size. Using lz4.frame.open(file).seek(0,2) takes a very long time (~13 s). I created the file using lz4.frame.open(file, 'wb',…
smp55
  • 403
  • 3
  • 8
1
vote
1 answer

Decompressing byte array compressed with LZ4 in JavaScript

I'm trying to decompress a byte array in JavaScript with lz4. I tried to implement the example in the npmjs page. I already have the data in compressed format, thus compressing it again leads to unwanted results. I have tried: var input =…
Dospacite
  • 63
  • 2
  • 7
1
vote
0 answers

Having malloc(): corrupted top size issue

I am trying to write code to test the speed of lz4 compression instead of using -b code in terminal. I am using ubuntu Ubuntu 20.04.2 LTS. And visual studio ide. Here is my code, it has some trouble when running the second time of the for loop in…
Ziruo Jin
  • 21
  • 3
1
vote
0 answers

Having error "undefinned reference to 'LZ4_compressBound' and 'LZ4_compress_default'

I try to use lz4 compress and decompress in my own C code. I include"lz4.h" from the downloaded file. But it still shows undefined references when I try to use the functions. I am using codeblocks, and I move the lz4.h file to the same folder of my…
Ziruo Jin
  • 21
  • 3
1
vote
0 answers

Prepraing lzo or lz4 files for Spark

I'm trying to choose the right format for file exchange with my spark application. I use Spark 2.4.7 + Haddop 2.10 on Kubernetess. My app downloads CSV file from S3 and process it. The file is provided by a 3rd party company. I was thinking about…
Matzz
  • 670
  • 1
  • 7
  • 17
1
vote
1 answer

How to decode last block in LZ4

I'm trying to understand how LZ4 compresses the last block in a sequence. According to LZ4 block format, there are specific rules required to terminate a block. Here is a small example: Raw Data (a new line character is included at the…
yildizabdullah
  • 1,895
  • 5
  • 24
  • 37
1
vote
2 answers

Combining methods to TAR + LZ4 in a single run in Java

I have two methods that separately TAR a set of files and then another that will compress it with LZ4. They both work fine, but I'm wondering if combining them together would be more efficient or save some time? Also I'm not really sure how I would…
Tacitus86
  • 1,314
  • 2
  • 14
  • 36
1 2 3
…
10 11