Call jpegOptim with RedirectStandardInput and RedirectStandardOutput

Question

I'm trying to do something that seems like it should be relatively simple: Call jpegoptim from C#.

I can get it to write to disk fine, but getting it to accept a stream and emit a stream has so far eluded me - I always end up with 0 length output or the ominous "Pipe has been ended."

One approach I tried:

var processInfo = new ProcessInfo(
    jpegOptimPath,
    "-m" + quality + " -T1 -o -p --strip-all --all-normal"
);
processInfo.CreateNoWindow = true;
processInfo.WindowStyle = ProcessWindowStyle.Hidden;
processInfo.UseShellExecute = false;
processInfo.RedirectStandardInput = true;
processInfo.RedirectStandardOutput = true;
processInfo.RedirectStandardError = true;

using(var process = Process.Start(processInfo))
{
    await Task.WhenAll(
        inputStream.CopyToAsync(process.StandardInput.BaseStream),
        process.StandardOutput.BaseStream.CopyToAsync(outputStream)
    );

    while (!process.HasExited)
    {
        await Task.Delay(100);
    }

    // Do stuff with outputStream here - always length 0 or exception
}

I've also tried this solution:

http://alabaxblog.info/2013/06/redirectstandardoutput-beginoutputreadline-pattern-broken/

using (var process = new Process())
{
    process.StartInfo.UseShellExecute = false;
    process.StartInfo.CreateNoWindow = true;
    process.StartInfo.RedirectStandardError = true;
    process.StartInfo.RedirectStandardOutput = true;
    process.StartInfo.FileName = fileName;
    process.StartInfo.Arguments = arguments;

    process.Start();

    //Thread.Sleep(100);

    using (Task processWaiter = Task.Factory.StartNew(() => process.WaitForExit()))
    using (Task<string> outputReader = Task.Factory.StartNew(() => process.StandardOutput.ReadToEnd()))
    using (Task<string> errorReader = Task.Factory.StartNew(() => process.StandardError.ReadToEnd()))
    {
        Task.WaitAll(processWaiter, outputReader, errorReader);

        standardOutput = outputReader.Result;
        standardError = errorReader.Result;
    }
}

Same problem. Output length 0. If I let jpegoptim run without the output redirect I get what I'd expect - an optimized file - but not when I run it this way.

There's gotta be a right way to do this?

Update: Found a clue - don't I feel sheepish - jpegoptim never supported piping to stdin until an experimental build in 2014, fixed this year. The build I have is from an older library, dated 2013. https://github.com/tjko/jpegoptim/issues/6

Chris Moschini · Answer 1 · 2015-11-06T13:31:15.107

A partial solution - see deadlock issue below. I had multiple problems in my original attempts:

You need a build of jpegoptim that will read and write to pipes instead of files-only. As mentioned builds prior to mid-2014 can't do it. The github "releases" of jpegoptim are useless zips of source, not built releases, so you'll need to look elsewhere for actual built releases.
You need to call it properly, passing --stdin and --stdout, and depending on how you'll be responding to it, avoid parameters that might cause it to write nothing, like -T1 (which will, when optimization is going to only be 1% or less, cause it to emit nothing to stdout).
You need to perform the non-trivial task of: Redirecting both input and output on the Process class
and avoiding a Buffer overflow on the input side that will get you 0 output once again - the obvious stream.CopyToAsync() overruns Process's very limited 4096 byte (4K) buffer and gets you nothing.

So many routes to nothing. None signalling why.

var processInfo = new ProcessInfo(
    jpegOptimPath,
    "-m" + quality + " --strip-all --all-normal --stdin --stdout",
);

processInfo.CreateNoWindow = true;
processInfo.WindowStyle = ProcessWindowStyle.Hidden;

processInfo.UseShellExecute = false;
processInfo.RedirectStandardInput = true;
processInfo.RedirectStandardOutput = true;
processInfo.RedirectStandardError = true;

using(var process = new Process())
{
    process.StartInfo = processInfo;

    process.Start();

    int chunkSize = 4096;   // Process has a limited 4096 byte buffer
    var buffer = new byte[chunkSize];
    int bufferLen = 0;
    var inputStream = process.StandardInput.BaseStream;
    var outputStream = process.StandardOutput.BaseStream;

    do
    {
        bufferLen = await input.ReadAsync(buffer, 0, chunkSize);
        await inputStream.WriteAsync(buffer, 0, bufferLen);
        inputStream.Flush();
    }
    while (bufferLen == chunkSize);

    do
    {
        bufferLen = await outputStream.ReadAsync(buffer, 0, chunkSize);
        if (bufferLen > 0)
            await output.WriteAsync(buffer, 0, bufferLen);
    }
    while (bufferLen > 0);

    while(!process.HasExited)
    {
        await Task.Delay(100);
    }

    output.Flush();

There are some areas for improvement here. Improvements welcome.

Biggest problem: On some images, this deadlocks on the outputStream.ReadAsync line.
It all belongs in separate methods to break it up - I unrolled a bunch of methods to keep this example simple.
There are a bunch of flushes that may not be necessary.
The code here is meant to handle anything that streams in and out. The 4096 is a hard limit that any Process will deal with, but the assumption that all the input goes in, then all the output comes out is likely a bad one and based on my research could result in a deadlock for other types of process. It appears that jpegoptim behaves in this (very buffered, very unpipe-like...) way when passed --stdin --stdout however, so, this code copes well for this specific task.

Call jpegOptim with RedirectStandardInput and RedirectStandardOutput

1 Answers1