2

I'm trying to configure a pool connections to my Go program with mongodb, where I set minPoolSize to 20 connections using mongodb go driver. Something like that:

cli, err := mongo.Connect(ctx, options.Client().ApplyURI(uri).SetMinPoolSize(20).SetMaxConnIdleTime(time.Second*5))

I did a test where my program reached 100 connections allowed. After 5 seconds, the number of connections dropped to 10. But it should have kept 20 connections. If I set minPoolSize to 5, it works ok.

Is 10 connections a limit to MinPoolSize for mongodb? How can I change it to keep 20 connections?

icza
  • 389,944
  • 63
  • 907
  • 827
Lawrence
  • 172
  • 1
  • 9
  • 1
    Have you also added `SetPoolMonitor` to your connection to print out what actually happens in the pool? (Not sure it'll help but we're using this in production to log connection events happening in the MongoDB driver—for we haven't found a better way to do that.) – kostix Feb 01 '22 at 16:49
  • No I haven't! But I will try to SetPoolMonitor. – Lawrence Feb 01 '22 at 18:31

1 Answers1

0

You set the max connection idle time to 5 seconds. Quoting from ClientOptions.SetMaxConnIdleTime():

SetMaxConnIdleTime specifies the maximum amount of time that a connection will remain idle in a connection pool before it is removed from the pool and closed. This can also be set through the "maxIdleTimeMS" URI option (e.g. "maxIdleTimeMS=10000"). The default is 0, meaning a connection can remain unused indefinitely.

The 5 seconds you set means that if a connection is in the pool and is not used for 5 seconds, it can be removed from the pool.

And you said you waited 5 seconds, and you experienced that connections in the pool were closed. This is the intended / allowed behavior.

If you want the connections to stay alive indefinitely, don't set the max connection idle time, or set 0 explicitly:

cli, err := mongo.Connect(ctx, options.Client()
    .ApplyURI(uri)
    .SetMinPoolSize(20)
    .SetMaxConnIdleTime(0))

You could ask what's the point of setting min pool size if connections can still be dropped from it. Yes, that's true, but only after the idle time has expired. The pool size may peak above the allowed minimum, e.g. if you set 20 as the minimum, and there's a need for a 100 connections, then 100 connections may be created. If all those 100 connections become idle, you can be sure at least 20 will be kept alive, for the idle time duration. If min pool size would be set to 10, you would have only guarantee for 10 being kept alive.

icza
  • 389,944
  • 63
  • 907
  • 827
  • actually I don't want to keep 100 connections alive indefinitly. After a while in idle state I want to drop the number of connections to a minimum pool size. It works when I define a minPoolSize to 10 and maxIdleTime to 5 seconds, it keep 10 connections alive. But more than 10 setted in minPoolSize, It just keep 10 connections – Lawrence Feb 01 '22 at 18:38
  • @Lawrence If you don't want 100 connections alive, set the max pool size, to a value you do want to keep. The default is 100. – icza Feb 01 '22 at 18:43
  • But as you said, the pool size may peak above the allowed minimum. If I set maxPool for less, I will not be prepared for peak. This makes sense for me to leave a peak margin, but then drop that number of unused connection while keeping a minimum for performance – Lawrence Feb 01 '22 at 18:48
  • @Lawrence Actually checking the doc of [`ClientOptions.SetMaxPoolSize()`](https://pkg.go.dev/go.mongodb.org/mongo-driver@v1.8.2/mongo/options#ClientOptions.SetMaxPoolSize) my previous comment was not true (removed it): _"SetMaxPoolSize specifies that maximum number of connections allowed in the driver's connection pool to each server. Requests to a server will block if this maximum is reached."_ – icza Feb 02 '22 at 07:58
  • 1
    @Lawrence Although I don't think you should be worried about this. Think about it: if you set e.g. 100 max pool size, that means you're allowed to have 100 simultaneous running operations. You say you won't be prepared for a peak, but even if you'd be allowed to have 1000 connections, then your MongoDB would be the bottleneck, and it's quite possible that wouldn't finish earlier than let's say serializing your operations on 100 connections. – icza Feb 02 '22 at 08:01
  • Thanks @icza! I will think about it! – Lawrence Feb 03 '22 at 13:03