4

I am running a server with 20 cpu cores and 96 GB of ram. I have configured Postgresql and Pgbouncer to handle 1000 connections at a time.

However when the connections increase (even though they are well below the 1000 limit I have set) I start getting failed connections. I checked the pgbouncer log and I noticed the following

ERROR accept() failed: Too many open files

What limit do I need to increase to solve this issue? I am running Debian 8

Arya
  • 8,473
  • 27
  • 105
  • 175
  • This is most likely about limits on Debian. Try to check these texts: https://github.com/pgbouncer/pgbouncer/issues/115 and https://www.postgresql.org/message-id/50DB32C7.50608@2ndQuadrant.com – JosMac Feb 24 '17 at 10:23
  • I agree it is usually because of ulimits. An easy way to check is looking at the /proc/{pid}/limits file to see what the ulimits of that process are. I am still getting this on my ubuntu server that has max files ulimit set to 100,000. When I reach about 1k connections it stops accepting them. The user accessing pgbouncer has 100k file max, the pgbouncer server has 100k file max and the postgres server also has the same max! I am a little confused... – hazmat Apr 28 '17 at 03:38
  • Wait, is postgresql configured for 1000 connections? The whole point of pgbouncer is that you can set postgresql to handle say 20 connections, while pgbouncer "funnels" thousands of connections on the client side to those 20 on the server side, thus preventing this exact kind of issue. – Scott Marlowe Aug 09 '17 at 17:07
  • 3
    Here is the final solution: I had the same problem, I increased the max number of open files, but it didn't work. There is a bug in pgbouncer start script. This thread has the answer: https://serverfault.com/questions/716982/how-to-raise-max-no-of-file-descriptors-for-daemons-running-on-debian-jessie/718978#718978?newreg=4c75aa4193a5470dbae958c337767d92 – Payam Moin Afshari Mar 11 '19 at 15:29

2 Answers2

2

Increase the operating system limit of the maximum number of open files for the user under which pgBouncer is running.

Vitalii Zurian
  • 17,858
  • 4
  • 64
  • 81
Laurenz Albe
  • 209,280
  • 17
  • 206
  • 263
0

I added the below parameters in the pgBouncer service. After that, pgbanch was run again. So problems was solved. The file limit size depend on your Linux file size. You checked your system with these codes.

  • cat /proc/sys/fs/file-max
  • ulimit -n
  • ulimit -Sn
  • ulimit -Hn

vim /lib/systemd/system/pgbouncer.service

[Service]
LimitNOFILE=64000
LimitNOFILESoft=64000
Cihan BARAN
  • 61
  • 1
  • 4
  • On Stack Overflow, the **how** is important, but much of the site's quality level comes from people going out of their way to explain **why**. While a correct _one-line-answer_ get the person who asked the question past whatever hurdle they might be facing, it doesn't do them or future visitors much good in the long run. See [Is there any benefit in code-only answers?](https://meta.stackexchange.com/a/148274/183937) – Steve Feb 17 '23 at 14:50