JsonSerializer failing to write to a GZipStream

Question

I'm trying to serialize very large object directly to a zip stream. I manage to do this by serializing to a file stream in an intermediate step, loading it back and then compressing it.

I've also tried compressing directly to a memory stream and it works. But when i use a GZipStream I'm always left with an "unfinished" object, the data is there it's correctly formatted up to the point where it ends unexpectedly. It's not for lack of flushing buffers since I've already tried flushing everything.

Simplified sample code:

internal static byte[] SerializeAndCompress(object objectToSerialize)
{
    using(var memStream = new MemoryStream())
    using (var zipStream = new GZipStream(memStream, CompressionMode.Compress, true))
    using (var streamWriter = new StreamWriter(zipStream))
    using (var jsonWriter = new JsonTextWriter(streamWriter))
    {
        var jsonSerializer = new JsonSerializer { ContractResolver = new CamelCasePropertyNamesContractResolver(), Formatting = Newtonsoft.Json.Formatting.None };

        jsonSerializer.Serialize(jsonWriter, objectToSerialize);
        jsonWriter.Flush();

        return memStream.ToArray();
    }
}

Thanks.

score 12 · Answer 1 · answered Sep 30 '15 at 16:19

Rather than flushing the writer, I suggest you close it completely. That way the GZipStream knows there's no more data to write and can add any appropriate checksums or whatever it needs to do.

You could either call Close explicitly, or put it between the closing parts of using statements:

using(var memStream = new MemoryStream())
{
    using (var zipStream = new GZipStream(memStream, CompressionMode.Compress, true))
    using (var streamWriter = new StreamWriter(zipStream))
    using (var jsonWriter = new JsonTextWriter(streamWriter))
    {
        var jsonSerializer = new JsonSerializer { ContractResolver = new CamelCasePropertyNamesContractResolver(), Formatting = Newtonsoft.Json.Formatting.None };

        jsonSerializer.Serialize(jsonWriter, objectToSerialize);
    }
    return memStream.ToArray();
}

Note that by the time you call ToArray, the MemoryStream will already be closed, but that's okay - the data will still be there.

Thank you, this seems to have solved the problem. I need to make more tests to make sure. It works for the huge object but for some reason not for the test code with smaller objects. — pnsm, Sep 30 '15 at 16:41

JsonSerializer failing to write to a GZipStream

1 Answers1