7

I've been having some thoughts on setting the appropriate pool size (in database.yml).

I saw a similar question at what is the use of the pool option in database.yml but needed to go further to understand this more.

Here is my setup. I have instances running Unicorn with 5 worker process'. I also have Sidekiq instances with concurrency set to 5 (I am asumming thats 5 concurrent threads).

What should I consider for my pool size? What are other factors should I take into consideration?

This is my understanding so far (as quoted from an engineer I was speaking to earlier):

Suppose you leave it at the default 5. It just means that, inside "a single process", the pool size is 5. The pool is per process. It is not system-wide so having a value of 5 does not mean that you are limited to 5 processes it just means that each process has its own pool, of size 5.

In summary, each instance has a db pool size of 5. This also means the db pool size setting is not app wide, but per process/instance. Which also means, since Sidekiq and Unicorn will run in its own instance. It will have its own db pool size of 5. Is this assumption correct?

At this point, I can assume that the default pool size of 5 is generally safe. Your thoughts?

PS. If you can share your pool size for your AWS RDS instances. That would be appreciated.

Community
  • 1
  • 1
Christian Fazzini
  • 19,613
  • 21
  • 110
  • 215
  • 2
    PostgreSQL will typically perform best with less than 100 actively working sessions actually running queries, unless there's a connection pooler in front to queue up queries. Keep that in mind. See http://wiki.postgresql.org/wiki/Number_Of_Database_Connections – Craig Ringer Jan 30 '14 at 23:59
  • Also active running queries should be in some relation to available threads for database. – frlan Feb 02 '14 at 00:00

1 Answers1

2

Your pool size in database.yml should be not less then max number of sidekiq/unicorn processes (not sum). Example, if unicorn running with 5 processes and sidekiq starting with 15 concurrency, you should set 15 pool size. Or, if unicorn with 7 and sidekiq with 5, you need 7 connections in database.yml.

Kroid
  • 1,081
  • 2
  • 12
  • 22
  • 1
    >Your pool size in database.yml should be not less then max number of sidekiq/unicorn processes. Where did you read about this restriction? – yaru Feb 24 '16 at 08:55