3

We recently started getting the following error on one of our nginx boxes:

2011/05/25 16:35:51 [alert] 3580#0: accept() failed (24: Too many open files)

Checking /etc/security/limits.conf, we have this:

*                soft    nofile          900000
*                hard    nofile          900000

but when we did cat /proc/{pid}/limits it showed the file limit being 1024. When we restarted nginx the issue was fixed and /proc/{pid}/limits showed 900000. I'm thinking this may have been caused because the machine was rebooted, and on boot-up nginx was started before limits were applied. However everything I've been reading about how limits and pam work suggest that this isn't really how ulimits work. Does anyone know what's going on here?

Edit: Sorry, should mention os and stuff. We're running CentOS with kernel 2.6.18-194.26.1.el5, and nginx 1.0.1

Mediocre Gopher
  • 803
  • 1
  • 13
  • 24

1 Answers1

3

This bug report seems to confirm your initial suspicion:

This is the wrong answer to the question "how do I set ulimits for a daemon process?" /etc/security/limits.d is only processed by pam_limits, which has no reason to be in the path for service startup.

On manual restart, on the other hand, nginx is inheriting the limits from your shell.

The workaround: just call ulimit in the nginx initscript. You can't use ulimit -n as a regular user, but the init script should have enough privileges when being run on startup.

Eduardo Ivanec
  • 14,881
  • 1
  • 37
  • 43
  • That's what we were thinking (the init script part, that is). In other places I've read that putting "session required /lib/security/pam_limits.so" in /etc/pam.d/login would also fix the issue. I like that idea better because it ensures other server processes would automatically get the ulimits without having to edit a ton of initscripts (assuming this method works). What do you think? – Mediocre Gopher May 27 '11 at 15:44
  • PAM is not the solution because PAM is only used during login and init scripts do not log in. – DerfK May 27 '11 at 19:46