nginx is cutting the end of dynamic pages and caching them

Question

I moved one of my old sites from an Apache to an nginx server. Everything is working fine but the site has some long content (a +100k generated HTML file).

My first trial was to disable chunked transfer encoding, but that did not help.

Here it is my nginx config:

$ cat /etc/nginx/nginx.conf
user www-data;
worker_processes  1;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;

    sendfile        on;
    keepalive_timeout  65;
    tcp_nodelay        on;

    gzip  on;
    gzip_static on;
    gzip_http_version   1.0;
    gzip_disable        "MSIE [1-6]\.";
    gzip_vary           on;
    gzip_comp_level 1;
    gzip_proxied    any;
    gzip_types      text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript text/x-js;
    gzip_buffers    16 8k;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

$ cat /etc/nginx/sites-enabled/example.com
server {
    listen   443 ssl;
    server_name example.com;

    access_log /var/log/nginx/example.com.access.log;
    error_log /var/log/nginx/example.com.error.log;

    charset iso-8859-2;
    root   /var/www/public/example.com;

    chunkin off;
    chunked_transfer_encoding off;

        location ~ ^.+\.php {
          fastcgi_split_path_info ^((?U).+\.php)(/?.+)$;
          fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
          fastcgi_param SCRIPT_NAME $fastcgi_script_name;
          fastcgi_param PATH_INFO $fastcgi_path_info;
          fastcgi_pass   127.0.0.1:9000;
          include        fastcgi_params;
        }

        location / {
          index index.php;
          try_files $uri /index.php?$args;
        }

    ssl_certificate     /etc/nginx/ssl/server.crt;
    ssl_certificate_key /etc/nginx/ssl/server.key;

}

There are some weird things going on. Firebug shows me the page is being cached, but I don't know why.

UPDATE:

Finally I can reproduce the issue by using the following PHP script:

<?php
//error_reporting(E_ALL ^ E_NOTICE ^ E_DEPRECATED); //the whole content printed as expected
error_reporting(E_ALL & ~E_DEPRECATED); //truncated content
header("Content-Type: text/plain; charset=iso-8859-2");

$i = 500000;
while ($i) {
  $i--;
  printf("%10s", $i);
  if (!($i%50)) {
    echo "\n";
  }
  $a = $undefined;
}

This script runs and terminates normally if I exclude E_NOTICE from error reporting.

I just opened a ticket http://trac.nginx.org/nginx/ticket/373 and it seems the I have problem with old version of PHP. — oliver nadj, Jun 11 '13 at 07:35

Kien Nguyen · Accepted Answer · 2013-10-08T04:21:49.203

7

Check permission of these directories:

client_body_temp_path
proxy_temp_path
fastcgi_temp_path

It is likely that one or all of these directories is not writable for nginx. These temp folders act as buffers for nginx when handling big requests, if nginx cannot write to them then nginx just returns the content which is stored in its memory.

You should also see errors similar to below in your error logs, if not check your log level / log path again.

2013/10/07 11:01:09 [crit] 3307#0: *33 open() "/var/lib/nginx/tmp/proxy/2/00/0000000002" failed (13: Permission denied) while reading upstream

edited Oct 08 '13 at 04:21

answered Oct 08 '13 at 04:13

Kien Nguyen

186
1
4

After searching for hours how to fix a Wordpress chunked Transfer Encoding Issue this has finally solved it. Bless! – flunder Jan 23 '15 at 17:43

score 1 · Answer 2 · answered Jun 07 '13 at 11:08

1

Possible connection with this bug http://www.ruby-forum.com/topic/4080504 , try updating nginx.

answered Jun 07 '13 at 11:08

Andrei Mikhaltsov

3,027
1
23
31

Thx @Andrei, I updated nginx to 1.4.1 but the issue is still exists. – oliver nadj Jun 10 '13 at 14:29
Does turning off gzip in nginx have any effect on size or anything? – Andrei Mikhaltsov Jun 10 '13 at 14:46
I already tried to disable gzipping, chunked ancoding and the ssl, but it did not help. – oliver nadj Jun 10 '13 at 15:07

score 0 · Answer 3 · answered Apr 30 '15 at 16:41

Not necessarily to do with html only, but also JS and CSS files, the sendfile option could be an issue too.

As reported in http://www.conroyp.com/2013/04/25/css-javascript-truncated-by-nginx-sendfile/ turning

sendfile off

can mean your files are not truncated

score 0 · Answer 4 · edited Jul 04 '16 at 12:14

0

The sendfile off solution is not working for me. I've fixed it by disabling the open_file_cache.

edited Jul 04 '16 at 12:14

Dave M

4,514
22
31
30

answered Jul 04 '16 at 09:39

Gaptek Update

61
1
3

nginx is cutting the end of dynamic pages and caching them

4 Answers4