Reversing InputStream/OutputStream (Bi-Directional) Operations

Question

I've observed that many libraries only support stream processing in one direction, such as compression, encryption, etc.

GZIPInputStream/GZIPOutputStream makes the assumption that you will only inflate-on-read (wrapping an InputStream) and deflate-on-write (wrapping an OutputStream).

Likewise, many examples of PGP via BouncyCastle only perform decrypt-on-read (again, wrapping an InputStream) and encrypt-on-write (again, wrapping an OutputStream)

Basing off a mix of examples I've encountered across SO and elsewhere, I'm trying to write a set of generic utility functions, which will allow (probably via piping with PipedInputStream/PipedOutputStream) the ability to support bi-directional stream processing, such as with GZIP:

• Inflate on read (supported out-of-the-box)
• Deflate on write (supported out-of-the-box)
• Inflate on write (missing; example use-case of inflating compressed data on-the-fly from a socket to disk)
• Deflate on read (missing; example use-case of deflating uncompressed data on-the-fly from disk to a socket)

For the case of running an InputStream through OutputStream processing (deflate on write), the following code works:

(see also: IOUtils)

static InputStream outputTransformingInputStream(InputStream inputStream, Function<OutputStream, OutputStream> transformer) {
    try {
        var pipedInputStream = new PipedInputStream();
        var pipedOutputStream = new PipedOutputStream(pipedInputStream);
        var transformerThread = new Thread(() -> {
            try (var transformerOutputStream = transformer.apply(pipedOutputStream)) {
                IOUtils.copy(inputStream, transformerOutputStream);
                transformerOutputStream.flush();
            } 
            catch (IOException exception) {
                throw new RuntimeException(exception);
            }
        });
        transformerThread.start();
        return pipedInputStream;
    }
    catch (IOException exception) {
        throw new RuntimeException(exception);
    }
}

Used with, for example:

// Deflating when reading from an InputStream
var inflatedInputStream = new FileInputStream("/path/to/file.txt");
var deflatingInputStream = outputTransformingInputStream(inflatedInputStream, GZIPOutputStream::new);
var deflatedBytes = deflatingInputStream.readAllBytes(); // Works 

The inverse approach isn't working (resulting in an empty byte[]) and perhaps it's just because I've been staring at streams and pipes too long and I've got the order out of whack:

(see also: IOUtils)

static OutputStream inputTransformingOutputStream(OutputStream outputStream, Function<InputStream, InputStream> transformer) {
    try {
        var pipedOutputStream = new PipedOutputStream();
        var pipedInputStream = new PipedInputStream(pipedOutputStream);
        var transformerThread = new Thread(() -> {
            try (var transformingInputStream = transformer.apply(pipedInputStream)) {
                IOUtils.copy(transformingInputStream, outputStream);
                outputStream.flush();
            }
            catch (IOException exception) {
                throw new RuntimeException(exception);
            }
        });
        transformerThread.start();
        return pipedOutputStream;
    }
    catch (IOException exception) {
        throw new RuntimeException(exception);
    }
}

And this example invocation is resulting in an empty inflatedBytes.

// Inflating when writing to an OutputStream
var inflatedOutputStream = new ByteArrayOutputStream();
var inflatingOutputStream = StreamFactory.inputTransformingOutputStream(inflatedOutputStream, GZIPInputStream::new);
inflatingOutputStream.write(deflatedBytes); // deflatedBytes previously deflated, see above
inflatingOutputStream.flush();
var inflatedBytes = inflatedOutputStream.toByteArray(); // Empty 

So, questions(s):

• Are the PipedInputStream/PipedOutputStream utility method examples the best approach for genericizing this functionality?
• If so, can someone tell me where I went wrong in the inputTransformingOutputStream function?
• If not, what would be a better approach to create generic support for bi-directional stream processing?

---

So my continued mention of GZIP and inflate/deflate probably looks like that's specifically what I'm trying to do.

More generically, given libraries Foo and Bar that do unidirectional transformations of stream data:

interface FooService {
     InputStream enfooify(InputStream inputStream);
     OutputStream defooify(OutputStream outputStream);
}

interface BarService {
    InputStream enbarify(InputStream inputStream);
    OutputStream debarify(OutputStream outputStream);
}

I'm trying to create a generic solution to:

• enfooify an OutputStream (on write), or
• debarify an InputStream (on read)
• etc.

Less about GZIP, or PGP, or anything in particular, and more about supporting generic bi-directional stream data transformation when a library doesn't already.

Answer 1

while I can share your observation, I do not see an issue here. Let's look at the two missing examples:

• Inflate on write (missing; example use-case of inflating compressed data on-the-fly from a socket to disk)
• Deflate on read (missing; example use-case of deflating uncompressed data on-the-fly from disk to a socket)

In both cases you need to read data, process (inflate/deflate) and write it. For these three steps it is of little effect whether you move the processing to the left side (read and inflate/deflate) or the right side (inflate/deflate and write). For the same reason it also is of little effect is one of them is left and the other one right.

You can definitely build the use cases you mentioned with the existing classes.