0

I am using pecl mongo 1.4.x driver(http://pecl.php.net/package/mongo/1.4.1), with the setup mentioned in the title on a moderate traffic services (5K - 10K request per min).

And I've found that, in mongoDB the Auth command is taking a large chunck of traffic, and connection request rate is like 30-50 per second.

This impacts the performance seriously(lock ratio up, memory management doesn't cops nicely)

And if I do netstat in a box(which I have 5-8 boxes in total), I see 2-3K mongo connections in total (some in WAIT some in ESTABLISHED) per box.

My question is how can I reduce the # of connection to mongoDB, how to setup persistent connection properly?

It seems the way of persistent connection working in PECL mongoDB Driver has been changing since 1.2 then 1.3 and it performs slightly differently in 1.4.

Here is the way I invoke the client with the driver:
$mongo = new MongoClient(
"host1:11004,host2:11004", array(
'replicaSet' => MG_REPLICASET,
'password'=>"superpasswd",
'username'=>"myuser",
'db'=>"mydb",
'journal' => true,
"readPreference"=> MongoClient::RP_SECONDARY_PREFERRED
) );

Leo Liang
  • 55
  • 1
  • 6
  • I don't see the place where you got instruction to do stuff, yes making a connection is a considerable job, also the connection should be persistent by default so I am unsure how/if your connections are being gced – Sammaye Jul 24 '13 at 17:00
  • A lot of WAIT state connection seems because of the small timeout limit. Workers get killed way too often. – Leo Liang Jul 30 '13 at 05:40

2 Answers2

3

In the 1.4 version all connections are persistent, unless you close them yourself - which you should never do. You will see a connection per IP/username/password/database combination from each PHP processing unit. In your case, per each PHPFPM process. In order to reduce the number of connections, you need to have less username/password/database combinations. However, with 8 boxes, and 50 FPM processes and 3 nodes in your replicaset you're already at 1200 connections - without even taken into account database/username/password differences. There is not much that you can do about that, but it shouldn't have a big influence in performance. You are much more likely to hit RAM/slow disk limitations.

Derick
  • 35,169
  • 5
  • 76
  • 99
0

I guess I found a solution to avoid exceessive mongo connection request.

We need to set PHP_FCGI_MAX_REQUESTS (or pm. max_requests in php-fpm) to a larger number, so the process won't recycle too often.

Also I need to make sure pm.request_terminate_timeout is not too small, so workers won't be killed too often.

Leo Liang
  • 55
  • 1
  • 6