Why does my GZip implementation result in invalid gzip

Question

I am attempting to gzip compress some data which is to be consumed by a separate app (a tilemap program called Tiled). I am unable to produce data which the app understands, and I believe it is because my zipped data is invalid. To verify, I have tested my zipped base64 string on https://codebeautify.org/gzip-decompress-online, and according to this site my gzip string is also invalid.

My implementation is:

uint[] values = new uint[4];
values[0] = 1;
values[1] = 1;
values[2] = 1;
values[3] = 1;

using var memoryStream = new MemoryStream();
using var gzipStream = new GZipStream(memoryStream, CompressionMode.Compress);
using var writer = new BinaryWriter(gzipStream);
for (int i = 0; i < values.Length; i++)
{
    writer.Write(values[i]);
}

writer.Flush();
gzipStream.Flush();
memoryStream.Position = 0;
var memoryBytes = memoryStream.ToArray();

var convertedString = Convert.ToBase64String(memoryBytes);

My resulting string is:

H4sIAAAAAAAACmJkYGBgRMIAAAAA//8=

When entered in the GZip website listed above, it returns a "Data Integrity Error"

However, if I attempt to decompress this in C#, I get the same data, so a "round trip" works.

For reference, the same data (1, 1, 1, 1) zipped through Tiled produces this string:

H4sIAAAAAAAACmNkYGBgRMIAUPFgrRAAAAA=

This string correctly decompresses in the website listed above too. Note that both strings decompress correctly using a GZipStream in C#, so the GZip implementation in .NET must be somehow more tolerant of whatever I'm doing wrong.

Can anyone tell me why my output string does not match the string produced by Tiled, why my string is not valid according to Tiled and that app, and how to fix this?

score 2 · Answer 1 · answered Aug 10 '21 at 02:21

As Mark mentioned, you need to use Close() to complete the gzip stream. In addition, you do not need to flush the writer or reset the memoryStream position. This will be handled by closing the GzipStream instance.

uint[] values = new uint[4];
values[0] = 1;
values[1] = 1;
values[2] = 1;
values[3] = 1;

using var memoryStream = new MemoryStream();
using var gzipStream = new GZipStream(memoryStream, CompressionMode.Compress);
using var writer = new BinaryWriter(gzipStream);
for (int i = 0; i < values.Length; i++)
{
    writer.Write(values[i]);
}

gzipStream.Close();

var memoryBytes = memoryStream.ToArray();
var convertedString = Convert.ToBase64String(memoryBytes);

score 1 · Accepted Answer · answered Aug 10 '21 at 01:40

Flush() forces the compression of the data provided so far, but does not complete the gzip stream. You need to use Close().

The first one is an incomplete gzip stream that ends in the middle of a deflate stream and has no gzip trailer. The second one is a complete gzip stream with a terminated deflate stream followed by a gzip trailer with a CRC and length that checks against the uncompressed data.

Why does my GZip implementation result in invalid gzip

2 Answers2