Nginx and fcgiwrap, incremental output problems

Question

When I run a cgi script under Apache any output is rapidly sent to the client. However when I run it under nginx with fcgiwrap it seems nothing is sent to the client until the script either finishes or produces a lot of output. In particular when using git-http-backend this leads to gateway timeouts on cloning large repositories (and lack of progress information on cloning smaller ones).

This behaviour can be seen with the following script.

 #!/bin/bash
 echo "Content-type: text/html"
 echo
 while :
 do
       echo this is a test.
       sleep 5
 done

Under apache the client will get some data every 5 seconds.

Under nginx with fcgiwrap I get no data and a gateway timeout.

nginx and fcgiwrap are packages from Debian Jessie. Versions 1.1.0-5 and 1.6.2-5+deb8u4

So the questions

1. Does anyone know which is responsible for this behaviour? nginx itself? fcgiwrap? both?
2. Is it something that can be fixed through configuration?
3. If it's a fcgiwrap problem are there alternative cgi wrappers available that don't suffer the problem?

Answer 1

Ok, figured it out. It seems both nginx and fcgiwrap have undesirable buffering in place. For nginx it seems it can be disabled through configuration while fcgiwrap needs a patch.

I have posted a debdiff for fcgiwrap at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863478 this adds a new option NO_BUFFERING

(update: Debian has now applied said patch, so if you are running Debian Buster or later you don't need to apply it yourself).

Then the nginx configuration needs changing to pass the NO_BUFFERING option to fcgiwrap and to disable buffering inside nginx.

To do this I added some settings immediately before and after the "include fastcgi_params;" line.

    #note: NO_BUFFERING relies on a patched fcgiwrap.
    fastcgi_param NO_BUFFERING 1;
    include fastcgi_params;
    gzip off;
    fastcgi_buffering off;