An Apache Bench test revealed that high throughput on the database caused the following...
- Apache threads stuck in "Sending Reply" state, all related to a particular PHP file (as seen in Apache extended status).
- MySQL sleeping connections with the same user as used by the PHP file.
Note: Apache Bench was used from a remote location as a mechanism for stressing the database only, ie a script just connected to the DB and ran 5 queries per load.
Running the same Apache Bench tests, but introducing a mysql_close() at the end of the script solved the problem. What I'd like to understand is why this happens.
A popular theory internally we have is that:
- The increased throughput on the database somehow prevented Apache from serving its requests properly. A buffer somewhere, either in MySQL or at the OS level got filled up.
- The Apache requests therefore got stuck in a Sending Reply state, and because there was no
mysql_close()at the end of the PHP file a database connection remained open - probably only to expire once MySQL's connection timeout limit was reached and the MySQL connection was closed. To be clear, we are not usingmysql_pconnect(), but we are usingmysql_connect().
As above, whilst we've 'solved' the problem we'd love to get to the bottom of this with a solid answer rather than an educated guess.
Anyone experienced this before and know of a solution? Any tricks/thoughts/methods of identifying this.
