GZipStream does gzipping but ungzipping file end up with "Unexpected end of data"

Question

Does anyone know why I'm getting the "Unexpected end of data" error message when un-gzipping the gzip file?

To verify the bytes data is not corrupted, I use the FooTest4.csv to write to file and was able to opened the file successfully.

Both 'FooTest3.csv.gzand 'FooTest2.csv.gz ran into "Unexpected end of data" when un-gzipping.

    public static List<byte> CompressFile(List<byte> parmRawBytes)
    {
        //Initialize variables...
        List<byte> returnModifiedBytes = null;

        File.WriteAllBytes(@"X:\FooTest4.csv", parmRawBytes.ToArray());

        using (var memoryStream = new MemoryStream())
        {
            using (var gzipStream = new GZipStream(memoryStream, CompressionMode.Compress, false))
            {
                gzipStream.Write(parmRawBytes.ToArray(), 0, parmRawBytes.ToArray().Length);
                gzipStream.Flush();
                File.WriteAllBytes(@"X:\FooTest3.csv.gz", memoryStream.ToArray());
                returnModifiedBytes = memoryStream.ToArray().ToList();
            }
        }

        File.WriteAllBytes(@"X:\FooTest2.csv.gz", returnModifiedBytes.ToArray());

        return returnModifiedBytes;
    }

You're calling `memoryStream.ToArray()` prematurely: `GZipStream.Dispose` writes things to its underlying stream. Remove call to `gzipStream.Flush` (which is moot) and pull the call to `memoryStream.ToArray()` from under the inner `using` block. — ach, Aug 11 '16 at 17:40
Pull the call to `memoryStream.ToArray()`, what do you mean by that? — fletchsod, Aug 11 '16 at 17:45
Place it inside the outer `using` block, but outside the inner: `using (var memoryStream = ...) { using (var gzipStream = ...) { gzipStream.Write(...); } memoryStream.ToArray(); }`. — ach, Aug 11 '16 at 17:49
That's it! It works now. I'm surprised it have to be outside for it to work, meaning gzipstream have to be closed first. — fletchsod, Aug 11 '16 at 17:53
There is nothing to be surprised with. GZip compression method is not additive, it must know exactly when all incoming data ends in order to emit compressed data. That's why `Flush` has no effect on `GZipStream`. Your having called `Flush` doesn't mean you're not going to insert more data later - and every single new byte to add may change the whole compressed data block. — ach, Aug 11 '16 at 18:01

score 6 · Accepted Answer · answered Apr 24 '17 at 23:28

6

GZipStream needs to be closed so it can write some terminating data to the end of the buffer to complete the gzip encoding.

byte[] inputBytes = ...;
using (var compressedStream = new MemoryStream())
{
    using (var compressor = new GZipStream(compressedStream, CompressionMode.Compress))
    {
        compressor.Write(inputBytes, 0, inputBytes.Length);
    }

    // get bytes after the gzip stream is closed 
    File.WriteAllBytes(pathToFile, compressedStream.ToArray());
}

answered Apr 24 '17 at 23:28

kalitsov

1,319
3
20
33

Giving you points so the old post can be closed. I don't remember what I did to make it work a year ago. :-) – fletchsod Apr 25 '17 at 13:14
1

If anyone is coming across this answer now, you'll find that in later .NET releases the inner using statement will cause the compressedStream to be closed once the compressor is disposed. This will cause an exception on the final File.WriteAllBytes call due to the stream being closed. The simple fix is to add the leaveOpen: true argument to the GZipStream constructor as such: `using (var compressor = new GZipStream(compressedStream, CompressionMode.Compress, leaveOpen: true))` – geeves Aug 17 '22 at 17:47

Pau C · Answer 2 · 2016-08-11T17:46:48.903

Instead of loading the bytes, compressing and saving them you could do do compression and writing at once. Also I don't know why you're using List<Byte> instead of byte[], maybe this could be it.

void CompressFile(string inputPath, string outputPath)
{
   Stream readStream = new FileStream(inputPath, Filemode.Open);
   Stream writeStream = new FileStream(outputPath, FileMode.Create);
   Stream compressionStream = new GZipStream(writeStream. CompressionMode.Compress);
   byte[] data = new byte[readStream.Length];
   readStream.Read(data, 0, data.Length);
   compressionStream.Write(data, 0, data.Length);
   readStream.Close();
   writeStream.Close();    
}

byte[] CompressFile(string inputPath)
{
  byte[] data = File.ReadAllBytes(inputPath);
  MemoryStream memStream = new MemoryStream(data);
  var gzipStream = new GZipStream(memStream, CompressionMode.Compress);
  gzipStream.Write(data, 0, data.Length);
  gzipStream.Close();
  return gzipStream.ToArray();
}

PS: I wrote the code in the text editor, so there might be errors. Also you say the error is on the "unzippiing", why no show us the unzip code?

But on your code you're writting to the disk. I'll edit the answer. — Pau C, Aug 11 '16 at 17:43
That part for debugging. The return value return the byte data. Oh well. — fletchsod, Aug 11 '16 at 17:44

GZipStream does gzipping but ungzipping file end up with "Unexpected end of data"

2 Answers2

Linked