Zlib uncompressed for partial

Question

Please help me understand what I'm doing wrong?

Suppose I have a compressed data buffer using Zlib of course. Now I want to decompress this data, but not in one piece, when I allocate a guaranteed size for the output buffer so that all the decompressed data fits there at once, but in parts, for example, 1 byte each.

I do like this: But inflate always throws a Z_BUF_ERROR error - that is, inflate cannot decompress data into a 1-byte buffer. And I can't figure out how to fix it.

If possible please explain.

#include <iostream>
#include <string>
#include <zlib.h>


static std::string in_buff_string;
static std::string out_buff_string;
static z_stream stream;



int my_uncompress_func__from_memory(unsigned long CHUNK_SIZE, char* char_p__to_compressed_data, unsigned long size__compressed_data)
{

    //char_p__to_compressed_data - buffer with compresed data
    //size__compressed_data      - size data in buffer "char_p__to_compressed_data "

    CHUNK_SIZE = 1;  //1 byte


    //-------------------------------------
    in_buff_string.resize(CHUNK_SIZE);           
    out_buff_string.resize(CHUNK_SIZE);         
    int err;
    //-------------------------------------

    stream = { 0 };


    //-------------------------------------
    err = inflateInit2(&stream, 15);

    if (err != Z_OK && err != Z_STREAM_END)
    {
        return 1;
    }
    //--------------------------------------




    //----------------------------------------------------
    int flush;
    size_t cntr = 0;
    //----------------------------------------------------


    do
    {
        if (cntr != size__compressed_data)
        {

            //----------------------------------------------
            memcpy(&in_buff_string[0], char_p__to_compressed_data + cntr, CHUNK_SIZE);  

            stream.avail_in = CHUNK_SIZE;
            stream.next_in = (Bytef*)&in_buff_string[0];

            cntr++;
            //----------------------------------------------
           
            flush = Z_NO_FLUSH;
        }
        else
        {
            flush = Z_FINISH;
        }




        do
        {

            stream.avail_out = CHUNK_SIZE;
            stream.next_out = (Bytef*)&out_buff_string[0];


            //------------------------------------------------
            err = inflate(&stream, flush);

            if (err == Z_BUF_ERROR)
            {
                std::cout << "Z_BUF_ERROR" << std::endl;
                return 1;
            }
            //------------------------------------------------

            int nbytes = CHUNK_SIZE - stream.avail_out;

            if (nbytes != 0)
            {
                std::cout << out_buff_string << std::endl;
            }



        } while (stream.avail_out == 0);


    } while (flush != Z_FINISH);


    inflateEnd(&stream);

    return 0;


}

I did everything like zpipe.c - why doesn't it want to work? :(

Compressing means you store many bytes in less bytes. How do you want to predict that your decompressed data chunk fits in one byte?! I think your requirement doesn't make much sense. Try using a larger output buffer. — harper, Dec 20 '22 at 18:58
@harper, why should i use a bigger buffer more than 1 byte ? I did as it says in zpipe.c. Have you ever used Zlib yourself? — Serg_, Dec 20 '22 at 19:03
[FAQ](https://github.com/madler/zlib/blob/cacf7f1d4e3d44d871b605da3b647f07d718623f/FAQ#L33) says: `When setting the parameter flush equal to Z_FINISH, also make sure that avail_out is big enough to allow processing all pending input.` Maybe, this is the case? — dewaffled, Dec 21 '22 at 02:17

score 1 · Accepted Answer · answered Dec 21 '22 at 03:18

Z_BUF_ERROR is not an error. All it means is that that call of inflate() was not able to consume any input nor deliver any output. That is inevitable in your loop, since avail_out will be zero every time a byte is delivered, assuring that you execute inflate() again until eventually it has nothing to do, so that avail_out is not zero. Only then will you exit the loop.

If you simply ignore Z_BUF_ERRORs, the inflation will complete successfully.

Other comments on your code:

You are not looking for Z_STREAM_END, which indicates the end of the compressed stream and successful decompression. If your data ends prematurely, or if the integrity check fails, you will not know. If the stream ends in before size__compressed_data bytes, then your code will oddly keep feeding bytes that are not actually consumed, and you will not know.
You are not checking for other errors, most notably Z_DATA_ERROR, which indicates invalid compressed data.
You do not need to give inflate() the Z_FINISH flush value. Just keep giving it Z_NO_FLUSH and it will detect the end of the stream on its own.
You are passing chunk as a parameter, and then overwriting it.
The stream = { 0 }; needs to be stream = {}; in order to zero out the entire structure.
Your std::cout << out_buff_string << std::endl; is odd, adding a new line after every character of output. It's also wrong for values of chunk greater than one, where the string of fixed length might have more characters than were actually delivered by the last inflate() call.

Many thanks for such a detailed answer. This is just an example, in the full code I check for all errors of course. And I don't have a stream, my "stream" ends where the Input buffer ends. — Serg_, Dec 21 '22 at 08:03
What you mean to say is that it is _supposed_ to end there. You always need to check to see that it actually _does_. There could be an error in transit. — Mark Adler, Dec 21 '22 at 16:27
in this case, my goal was: to decompress the compressed file in parts, since the file size is quite large, I do not want to allocate the entire guaranteed size for the output buffer at once and decompress it in small parts, that is, I know exactly when my Input buffer ends. — Serg_, Dec 21 '22 at 18:51

Zlib uncompressed for partial

1 Answers1