0

On my server, /dev/root is getting full very fast after each reboot: a munin graph showing it

if i type

sudo lsof | grep "deleted"

i get:

nginx     6012        www-data   44u      REG                9,1    5917025     264136 /var/lib/nginx/fastcgi/9/06/0000000069 (deleted)
nginx     6015        www-data   47u      REG                9,1   11531249     264134 /var/lib/nginx/fastcgi/8/06/0000000068 (deleted)
nginx     6021        www-data   44u      REG                9,1 1073744881     140088 /var/lib/nginx/fastcgi/9/00/0000000009 (deleted)
nginx     6021        www-data   47u      REG                9,1    1162123     264127 /var/lib/nginx/fastcgi/1/06/0000000061 (deleted)
nginx     6021        www-data   57u      REG                9,1    9035129     264131 /var/lib/nginx/fastcgi/6/06/0000000066 (deleted)
nginx     6021        www-data   58u      REG                9,1  158384950     264124 /var/lib/nginx/fastcgi/9/05/0000000059 (deleted)
nginx     6021        www-data   64u      REG                9,1  253916765     264088 /var/lib/nginx/fastcgi/3/04/0000000043 (deleted)
nginx     6021        www-data   72u      REG                9,1   65144633     264039 /var/lib/nginx/fastcgi/8/03/0000000038 (deleted)
nginx     6025        www-data   47u      REG                9,1  329180930     263905 /var/lib/nginx/fastcgi/6/04/0000000046 (deleted)
nginx     6028        www-data   43u      REG                9,1  703946757     264108 /var/lib/nginx/fastcgi/7/04/0000000047 (deleted)
nginx     6028        www-data   47u      REG                9,1  102594620     264035 /var/lib/nginx/fastcgi/2/03/0000000032 (deleted)
nginx     6028        www-data   53u      REG                9,1  214559921     264083 /var/lib/nginx/fastcgi/9/03/0000000039 (deleted)
nginx     6028        www-data   59u      REG                9,1  175354461     262715 /var/lib/nginx/fastcgi/2/05/0000000052 (deleted)
nginx     6028        www-data   80u      REG                9,1    9211086     264089 /var/lib/nginx/fastcgi/4/05/0000000054 (deleted)
nginx     6031        www-data   41u      REG                9,1  103114289     264125 /var/lib/nginx/fastcgi/3/06/0000000063 (deleted)
nginx     6031        www-data   47u      REG                9,1  144005653     263762 /var/lib/nginx/fastcgi/5/01/0000000015 (deleted)
nginx     6031        www-data   48u      REG                9,1  359997771     264084 /var/lib/nginx/fastcgi/3/05/0000000053 (deleted)
nginx     6034        www-data   34u      REG                9,1  280936251     140071 /var/lib/nginx/fastcgi/4/00/0000000004 (deleted)
nginx     6034        www-data   57u      REG                9,1  119430446     264048 /var/lib/nginx/fastcgi/6/03/0000000036 (deleted)
nginx     6035        www-data   34u      REG                9,1 1073740797     263765 /var/lib/nginx/fastcgi/5/02/0000000025 (deleted)
nginx     6035        www-data   65u      REG                9,1   67211095     263917 /var/lib/nginx/fastcgi/7/03/0000000037 (deleted)
nginx     6036        www-data   28u      REG                9,1    2430656     263530 /var/lib/nginx/fastcgi/2/01/0000000012 (deleted)
nginx     6037        www-data   30u      REG                9,1    2749827     264110 /var/lib/nginx/fastcgi/7/05/0000000057 (deleted)
nginx     6038        www-data   55u      REG                9,1   34957948     263914 /var/lib/nginx/fastcgi/8/01/0000000018 (deleted)
nginx     6038        www-data   59u      REG                9,1   32122186     264086 /var/lib/nginx/fastcgi/2/04/0000000042 (deleted)
nginx     6038        www-data   66u      REG                9,1   52218002     264123 /var/lib/nginx/fastcgi/8/05/0000000058 (deleted)
nginx     6038        www-data   72u      REG                9,1   54845063     264135 /var/lib/nginx/fastcgi/7/06/0000000067 (deleted)
nginx     6038        www-data   78u      REG                9,1    2266978     140079 /var/lib/nginx/fastcgi/6/00/0000000006 (deleted)
nginx     6038        www-data   88u      REG                9,1  447169649     140011 /var/lib/nginx/fastcgi/2/00/0000000002 (deleted)
nginx     6038        www-data  130u      REG                9,1  121233674     264037 /var/lib/nginx/fastcgi/6/02/0000000026 (deleted)
nginx     6041        www-data   18u      REG                9,1   64162555     264038 /var/lib/nginx/fastcgi/0/03/0000000030 (deleted)
nginx     6041        www-data   46u      REG                9,1    1726840     264128 /var/lib/nginx/fastcgi/2/06/0000000062 (deleted)
nginx     6042        www-data   18u      REG                9,1   10279101     264121 /var/lib/nginx/fastcgi/0/05/0000000050 (deleted)
nginx     6043        www-data   28u      REG                9,1   93109363     264119 /var/lib/nginx/fastcgi/9/04/0000000049 (deleted)
nginx     6043        www-data   32u      REG                9,1   83795966     264129 /var/lib/nginx/fastcgi/5/06/0000000065 (deleted)
nginx     6045        www-data   10u      REG                9,1  308205332     264122 /var/lib/nginx/fastcgi/4/06/0000000064 (deleted)

what can i do to get rid of these files in /var/lib/nginx/fastcgi/ ?

even if i restart nginx, the same thing happens again and again.

nginx version: nginx/1.8.0

php5-fpm

ubuntu server 14.04

Update: /var/log/nginx/error/log

2015/11/26 17:05:29 [crit] 6038#0: *25448 writev() "/var/lib/nginx/fastcgi/4/06/0000001064" has written only 6333 of 8184 while reading upstream, client: 81.249.167.106, server: example.com, request: "POST /download/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "example.com", referrer: "http://example.com/"
2015/11/26 17:05:29 [crit] 6043#0: *25310 writev() "/var/lib/nginx/fastcgi/1/06/0000001061" has written only 212 of 8184 while reading upstream, client: 41.107.198.43, server: example.com, request: "POST /download/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "example.com", referrer: "http://example.com/"
2015/11/26 17:05:29 [crit] 6041#0: *24843 writev() "/var/lib/nginx/fastcgi/1/04/0000001041" has written only 1104 of 8184 while reading upstream, client: 45.219.155.237, server: example.com, request: "POST /download/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "example.com", referrer: "http://example.com/"
hedi
  • 101
  • 3
  • 1
    Is there some error logs ? – Dom Nov 26 '15 at 15:53
  • @Dom yes, i added error logs on my question – hedi Nov 26 '15 at 16:14
  • What are the first logs just after the launch of nginx ? – Dom Nov 26 '15 at 17:02
  • @Dom i don't know if it's just after or just before naginx restart: `2015/11/26 19:29:46 [emerg] 22886#0: bind() to 0.0.0.0:80 failed (98: Address already in use) 2015/11/26 19:29:46 [emerg] 22886#0: bind() to [::]:80 failed (98: Address already in use)` – hedi Nov 26 '15 at 18:31
  • Probably you have typos in config. Check all `error_log`/`access_log` directives. What's in that `/dev/root` file? There is no such file in normal system. My guess, someone has accidentally type `/dev/root` instead of `/dev/null`… – Alexey Ten Nov 27 '15 at 06:38
  • @AlexeyTen no typo in config: nginx -t returns ok. /dev/root is my system root partition, not a file. – hedi Nov 27 '15 at 10:05

1 Answers1

0

I solved my problem adding "X-Accel-Buffering: no" (it turns off fastcgi_buffering and proxy_buffering) to the script that was the cause of the problem (sending big files via php).

hedi
  • 101
  • 3