0

I have a perl and shell script that process a bunch of data and output results as they happen, like:

5 processed in 0.58 seconds.
10 processed in 0.79 seconds.
...
150 processed in 0.65 seconds. DONE!

etc.

However, when I visit the URL with Chrome, it shows nothing, waits until the script finishes, then shows the entire output at once. Is there a way to configure nginx to show each line output as it happens? I used to be able to do this with Apache.

I put gzip off in the nginx config, thinking this could nginx waiting to compress the text before sending it to the client, but that did not help.

I also tried buffer-flushing techniques in the scripts themselves, like these lines at the top of my Perl scripts:

use IO::Handle;
$| ++;
STDERR->autoflush(1);
STDOUT->autoflush(1);
print "Content-type: text/plain\n\n";

Relevant parts of the nginx config look like this

  ssl on;
  root /my/path/cgi ;
  proxy_read_timeout 900s ;
  fastcgi_read_timeout 900s ;
  fastcgi_request_buffering off ;
  gzip off ;
  location ~ \.pl|cgi$ {
    try_files $uri =404;
    gzip off;
    fastcgi_pass  unix:/var/run/fcgiwrap.socket;
    include /etc/nginx/fastcgi_params;
    fastcgi_param SCRIPT_FILENAME  $document_root$fastcgi_script_name;
  }

Any ideas?

Crash Override
  • 601
  • 1
  • 10
  • 21
  • It might be that `fcgiwrap` implements some buffering. – Tero Kilkanen Aug 25 '20 at 06:34
  • Yeah from the looks of it, seems to be no way to turn off buffering for fastcgi. Instead, I set up an apache server listening on localhost with ExecCGI enabled for the cgi directory and `AddHandler cgi-script` specified, then I changed my nginx to `proxy_pass` the `.pl` and `.cgi` files to the local apache port. This worked. Apache does no caching of cgi output, and outputs the lines as they happen to the client. – Crash Override Aug 25 '20 at 18:18

1 Answers1

0

You need to tell nginx to not buffer the response from the FastCGI server, with fastcgi_buffering off;.

From the docs:

When buffering is disabled, the response is passed to a client synchronously, immediately as it is received. nginx will not try to read the whole response from the FastCGI server. The maximum size of the data that nginx can receive from the server at a time is set by the fastcgi_buffer_size directive.

Which you may also need to tweak.

But in general, a plain HTML web page is probably not the best interface for whatever it is you're doing.

Michael Hampton
  • 244,070
  • 43
  • 506
  • 972
  • Sadly this did not help. Same result. I also checked that this is not a browser issue by doing an HTTP GET on a linux shell and watching the raw output show nothing until the script finishes. – Crash Override Aug 24 '20 at 23:13
  • Also just tried adding http header `X-Accel-Buffering: no` to the output of the cgi script, also did not help. Docs for that https://www.nginx.com/resources/wiki/start/topics/examples/x-accel/#x-accel-buffering – Crash Override Aug 24 '20 at 23:37