C# reuse GZipStream for more than one decompressions

Question

I need to compress and decompress millions of strings individually. The first loop works. The second doesn't. Basically I don't know how to use streams. How do I get the second method, the one that reuses streams, to work?

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace ConsoleApplication1 {
    class Program {
        static void Main(string[] args) {
            string s = "Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah ";
            for (int i=0; i<10000; ++i) {
                var aCompressed = Zip(Encoding.ASCII.GetBytes(s));
                var aDecompressed = UnZip(aCompressed, 0, aCompressed.Length);
                string sDbg = System.Text.Encoding.ASCII.GetString(aDecompressed);
                Debug.Assert(sDbg == s);
            }
            // This loop is an utter failure.
            var UnZipper = new CUnZip(); // attempt to setup all the unzip framework once
            for (int i = 0; i < 10000; ++i) {
                var aCompressed = Zip(Encoding.ASCII.GetBytes(s));
                var aDecompressed = UnZipper.UnZip(aCompressed, 0, aCompressed.Length);
                string sDbg = System.Text.Encoding.ASCII.GetString(aDecompressed);
                Debug.Assert(sDbg == s);
            }
        }
        static byte[] Zip(byte[] aIn) {
            using (var outStream = new MemoryStream()) {
                using (var tinyStream = new GZipStream(outStream, CompressionMode.Compress))
                using (var mStream = new MemoryStream(aIn))
                    mStream.CopyTo(tinyStream);
                return outStream.ToArray();
            }
        }
        static byte[] UnZip(byte[] aIn, int i0, int cb) {
            using (var inStream = new MemoryStream(aIn, i0, cb))
            using (var bigStream = new GZipStream(inStream, CompressionMode.Decompress))
            using (var bigStreamOut = new MemoryStream()) {
                bigStream.CopyTo(bigStreamOut);
                return bigStreamOut.ToArray();
            }
        }
        // this class is an utter failure
        class CUnZip {
            GZipStream bigStream;
            MemoryStream inStream, bigStreamOut;
            BinaryWriter argh;
            public CUnZip() {
                inStream = new MemoryStream();
                argh = new BinaryWriter(inStream);
                bigStream = new GZipStream(inStream, CompressionMode.Decompress);
                bigStreamOut = new MemoryStream();
            }
            public byte[] UnZip(byte[] aIn, int i0, int cb) {
                argh.Write(aIn, i0, cb);
                argh.Flush();
                bigStream.CopyTo(bigStreamOut);
                return bigStreamOut.ToArray();
            }
        }
    }
}

Define doesn't work. Wrong results? Errors? Something else? But simply said: you can't just shove more data into the decompression and assume it'll happily comply and do what you want. You have to create new ones with fresh state. — Sami Kuhmonen, Sep 24 '16 at 03:35
Please point to specific details, instead of assuming we will be able to guess, first and second loops and point of failures as perceived by you — Mrinal Kamboj, Sep 24 '16 at 03:37
`bigStream = new GZipStream(inStream, ...` Why are you using `inStream` there. For fun? — Orkhan Alikhanov, Sep 24 '16 at 04:24
@Orkhan. This is the first time i have used streams. I did it because I don't know better. — johnnycrash, Sep 24 '16 at 18:20
@johnnycrash: Hmm, I missed that you are writing into `instream` using `BinaryWriter` — Orkhan Alikhanov, Sep 24 '16 at 18:53
@orkhan any ideas how to do it any differently? Can i use the BinaryWriter to write into the GZipStream? — johnnycrash, Sep 24 '16 at 19:18

score 1 · Accepted Answer · answered Sep 24 '16 at 04:49

1

Tested your code following will resolve the issue:

inStream.Position = 0;

Set it post argh.Flush() in the class CUnZip - UnZip Method

Details:

As you are reusing the MemoryStream in the CUnZip class, its position is set to the last byte written and to do any operation, that needs it to be read, you need to reset the position to the beginning (0), else it has nothing to read, therefore even when stream contained the data it cannot be read, as it was pointing to the end of stream

answered Sep 24 '16 at 04:49

Mrinal Kamboj

11,300
5
40
74

The gzip stream might have some internal state that cannot be reset. – usr Sep 24 '16 at 08:36
Reusing everything made unzipping a LOT faster. – johnnycrash Sep 24 '16 at 20:00

C# reuse GZipStream for more than one decompressions

1 Answers1