Load average 1.00 while top seems to show the server as idle

Question

My server is not responding to HTTP requests, and it took me a long time before being able to log in with SSH; it all looks like the server is overloaded and on its knees.

It already did that a few days ago, but after a reboot everything was back to normal. Today before rebooting, I'd like to understand what's happening.

I've finally been able to log in, and the load average shows a steady 1.00:

# uptime
15:00:46 up 4 days, 19:47,  1 user,  load average: 1.00, 1.00, 1.00

But top seems to say that the server is idle:

top - 15:01:52 up 4 days, 19:48,  1 user,  load average: 1.00, 1.00, 1.00
Tasks: 704 total,   1 running, 703 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3%us,  1.1%sy,  0.0%ni, 98.4%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:   4034032k total,  4009704k used,    24328k free,    16612k buffers
Swap:  1959920k total,    58576k used,  1901344k free,    22860k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
19137 root      16   0 13152 1748  936 R  2.9  0.0   0:00.27 top
    1 root      15   0 10364  744  620 S  0.0  0.0   0:03.26 init
    2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.02 migration/0
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.65 ksoftirqd/0
    4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
    5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 migration/1
    6 root      34  19     0    0    0 S  0.0  0.0   0:00.21 ksoftirqd/1
    7 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/1
    8 root      10  -5     0    0    0 S  0.0  0.0   0:04.73 events/0
    9 root      10  -5     0    0    0 S  0.0  0.0   0:02.29 events/1
   10 root      10  -5     0    0    0 S  0.0  0.0   0:00.01 khelper
   47 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 kthread
   52 root      10  -5     0    0    0 S  0.0  0.0   0:00.02 kblockd/0
   53 root      10  -5     0    0    0 S  0.0  0.0   0:00.05 kblockd/1
   54 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kacpid
  165 root      11  -5     0    0    0 S  0.0  0.0   0:00.00 cqueue/0
  166 root      11  -5     0    0    0 S  0.0  0.0   0:00.00 cqueue/1

Oh, and I've also seen the following messages in httpd's error_log:

PHP Fatal error: SQLSTATE[08004] [1040] Too many connections in ...

But access_log shows no requests for the last 8 hours, as all HTTP connections end up in a timeout...

Does anyone have an explanation for that?

Hard drive perhaps? Take a look at your smart attributes and maybe `iotop`. — Nathan C, Oct 08 '13 at 13:20
Also, you have 700+ processes. Have a look at `ps aux` to see what's running. — Nathan C, Oct 08 '13 at 13:21
The hard drive was simply full, I should have checked this first. Problem solved, thanks guys for your comments! — BenMorel, Oct 08 '13 at 14:00

BenMorel · Accepted Answer · 2013-10-08T14:00:01.533

1

The /var partition was simply full, hence the SQL errors as this is the partition where the mysql data is stored.

The main reason was that binary logging was enabled even though I wasn't using it for replication or anything, so I disabled this, removed the binary logs, and everything is back to normal!

edited Oct 08 '13 at 14:00

answered Oct 08 '13 at 13:54

BenMorel

4,507
10
57
85

Load average 1.00 while top seems to show the server as idle

1 Answers1