Ubuntu server using too much memory

Question

my server is at a very high memory usage constantly - to the point that it won't let me log in via ssh some times. I have to reboot the server before I can log in and when it's restarted the memory drops to ~90% usage and slowly works it's way up to ~190%.

I've noticed that restarting apache also puts me back to 90% and I can slowly watch the memory rise using hosting tools. I've been watching top, but i'm not sure what to look for. I'm pretty sure the culprit is apache, but I'm not sure why.

top at 130%

top - 08:14:59 up 49 min,  1 user,  load average: 0.04, 0.01, 0.00
Tasks:  41 total,   1 running,  40 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    946344k total,   724484k used,   221860k free,        0k buffers
Swap:        0k total,        0k used,        0k free,        0k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                   
    1 root      15   0 23456 1664 1276 S    0  0.2   0:01.82 init                                                       
 1173 messageb  16   0 23436  920  612 S    0  0.1   0:00.01 dbus-daemon                                                
 1222 root      15   0 49272 1088  540 S    0  0.1   0:00.02 sshd                                                       
 1237 root      18   0 27128 1800 1428 S    0  0.2   0:00.00 vsftpd                                                     
 1245 root      18   0 21088 1020  780 S    0  0.1   0:00.00 cron                                                       
 1249 mysql     15   0  178m  36m 7356 S    0  4.0   0:01.26 mysqld                                                     
 1285 syslog    18   0 12460  804  612 S    0  0.1   0:00.00 syslogd                                                    
 1305 bind      18   0  286m  26m 2056 S    0  2.8   0:00.02 named                                                      
 1328 root      21   0  6292  500  392 S    0  0.1   0:00.00 courierlogger                                              
 1329 root      18   0 18788  812  652 S    0  0.1   0:00.00 authdaemond                                                
 1339 root      18   0 18788  276  112 S    0  0.0   0:00.00 authdaemond                                                
 1340 root      18   0 18788  276  112 S    0  0.0   0:00.00 authdaemond                                                
 1341 root      18   0 18788  276  112 S    0  0.0   0:00.00 authdaemond                                                
 1341 root      18   0 18788  276  112 S    0  0.0   0:00.00 authdaemond                                                
 1342 root      18   0 18788  276  112 S    0  0.0   0:00.00 authdaemond                                                
 1343 root      18   0 18788  276  112 S    0  0.0   0:00.00 authdaemond                                                
 1426 root      15   0 37216 2276 1784 S    0  0.2   0:00.00 master                                                     
 1442 postfix   18   0 39440 2236 1760 S    0  0.2   0:00.00 qmgr                                                       
 1443 postfix   15   0 39280 2188 1728 S    0  0.2   0:00.00 pickup                                                     
 1559 root      18   0  120m 3928 2444 S    0  0.4   0:00.06 console-kit-dae                                            
 1650 root      16   0 79148 3464 2696 S    0  0.4   0:00.03 sshd                                                       
 1664 robert    17   0 79148 1744  960 S    0  0.2   0:00.01 sshd                                                       
 1665 robert    16   0 12476  904  736 S    0  0.1   0:00.01 sftp-server                                                
 1666 root      16   0 79120 3516 2732 S    0  0.4   0:00.03 sshd                                                       
 1680 jackson   15   0 79120 1732  932 S    0  0.2   0:00.14 sshd                                                       
 1681 jackson   15   0 18032 2140 1500 S    0  0.2   0:00.04 bash                                                       
 1946 root      18   0  227m  10m 4960 S    0  1.1   0:00.03 apache2                                                    
 1950 www-data  15   0  270m  52m 3676 S    0  5.7   0:01.34 apache2                                                    
 1951 www-data  15   0  227m 6828 1232 S    0  0.7   0:00.03 apache2                                                    
 1952 www-data  15   0  227m 6828 1232 S    0  0.7   0:00.03 apache2                                                    
 1960 www-data  15   0  227m 7056 1288 S    0  0.7   0:00.02 apache2                                                    
 1963 www-data  15   0  271m  53m 3696 S    0  5.8   0:01.30 apache2                                                    
 1964 www-data  15   0  270m  52m 3672 S    0  5.7   0:01.35 apache2                                                    
 1965 www-data  15   0  270m  52m 3672 S    0  5.7   0:01.21 apache2                                                    
 1968 root      16   0 79148 3468 2696 S    0  0.4   0:00.04 sshd                                                       
 1982 robert    18   0 79148 1752  960 S    0  0.2   0:00.00 sshd                                                       
 1983 robert    18   0 12476  924  732 S    0  0.1   0:00.00 sftp-server                                                
 1984 www-data  15   0  227m 6824 1228 S    0  0.7   0:00.02 apache2                                                    
 1985 www-data  15   0  227m 6824 1228 S    0  0.7   0:00.02 apache2                                                    
 1986 www-data  15   0  227m 6816 1224 S    0  0.7   0:00.00 apache2                                                    
 1987 www-data  18   0  227m 5908  560 S    0  0.6   0:00.00 apache2

free -m

             total       used       free     shared    buffers     cached
Mem:           924        697        227          0          0          0
-/+ buffers/cache:        697        227
Swap:            0          0          0

What should I do next? Thanks for the help.

** EDIT **

top (pushed > once)

top - 09:46:06 up  2:21,  1 user,  load average: 0.01, 0.03, 0.00
Tasks:  37 total,   1 running,  36 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    946344k total,   799432k used,   146912k free,        0k buffers
Swap:        0k total,        0k used,        0k free,        0k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
 3274 www-data  15   0  273m  54m 4756 S    0  5.9   0:02.84 apache2             
 2043 www-data  18   0  272m  54m 3692 S    0  5.8   0:03.65 apache2             
 3109 www-data  15   0  271m  53m 3748 S    0  5.8   0:05.44 apache2             
 3217 www-data  15   0  270m  52m 3700 S    0  5.7   0:01.46 apache2             
 3310 www-data  15   0  270m  52m 3672 S    0  5.7   0:01.29 apache2             
 2044 www-data  15   0  264m  46m 3764 S    0  5.0   0:03.20 apache2             
 1249 mysql     15   0  244m  38m 7348 S    0  4.2   0:03.35 mysqld              
 1305 bind      18   0  286m  26m 2060 S    0  2.8   0:00.02 named               
 2038 root      18   0  227m  10m 4960 S    0  1.1   0:00.06 apache2             
 3376 www-data  15   0  227m 7220 1376 S    0  0.8   0:00.00 apache2             
 3291 www-data  15   0  227m 7024 1376 S    0  0.7   0:00.02 apache2             
 3377 www-data  15   0  227m 6824 1228 S    0  0.7   0:00.00 apache2             
 3379 www-data  19   0  227m 5924  564 S    0  0.6   0:00.00 apache2             
 1559 root      18   0  120m 3928 2444 S    0  0.4   0:00.18 console-kit-dae     
 3358 root      16   0 79120 3508 2732 S    0  0.4   0:00.03 sshd                
 1650 root      15   0 79148 3476 2696 S    0  0.4   0:00.04 sshd                
 1426 root      15   0 37216 2276 1784 S    0  0.2   0:00.00 master              
 1442 postfix   15   0 39440 2236 1760 S    0  0.2   0:00.00 qmgr                
 3265 postfix   15   0 39280 2192 1728 S    0  0.2   0:00.00 pickup              
 3373 jackson   18   0 17976 2028 1448 S    0  0.2   0:00.00 bash                
 1664 robert    15   0 79148 1900 1060 S    0  0.2   0:00.21 sshd                
 1237 root      18   0 27128 1800 1428 S    0  0.2   0:00.00 vsftpd              
 3372 jackson   15   0 79120 1724  932 S    0  0.2   0:00.01 sshd                
    1 root      15   0 23456 1664 1276 S    0  0.2   0:01.82 init                
 3380 jackson   15   0 19100 1344 1072 R    0  0.1   0:00.06 top                 
 1222 root      15   0 49272 1088  540 S    0  0.1   0:00.06 sshd                
 1245 root      18   0 21088 1020  780 S    0  0.1   0:00.00 cron                
 1665 robert    15   0 12476  952  736 S    0  0.1   0:00.21 sftp-server         
 1173 messageb  16   0 23436  920  612 S    0  0.1   0:00.02 dbus-daemon         
 1329 root      18   0 18788  812  652 S    0  0.1   0:00.00 authdaemond         
 1285 syslog    18   0 12460  804  612 S    0  0.1   0:00.00 syslogd             
 1328 root      21   0  6292  500  392 S    0  0.1   0:00.00 courierlogger       
 1339 root      18   0 18788  276  112 S    0  0.0   0:00.00 authdaemond         
 1340 root      18   0 18788  276  112 S    0  0.0   0:00.00 authdaemond         
 1341 root      18   0 18788  276  112 S    0  0.0   0:00.00 authdaemond         
 1342 root      18   0 18788  276  112 S    0  0.0   0:00.00 authdaemond         
 1343 root      18   0 18788  276  112 S    0  0.0   0:00.00 authdaemond

-Zoredache I'm on a host and they charge a premium for their memory.. -SameKrieg In some spots, but I get these errors even when the php hasn't been hit yet. -voretaq7 I added an edit with what I think you're looking for — Jacksonkr, Dec 01 '11 at 17:49
@Jackson - you'd want to use the @ symbol if you want to address a specific user as a "paging" note (I used it with your name here even though I think you'll get pinged about a comment to your question regardless.) — Bart Silverstrim, Dec 01 '11 at 18:05
@Jackson - Yup, that's what I was looking for. Apache is definitely your heavyweight (you may want to look for memory leaks if it seems like they keep growing), but as everyone pointed out in the answers you definitely need to spend the cash for more RAM. — voretaq7, Dec 01 '11 at 19:25
You can use the '>' key to change the sort order in top, which will allow you to sort by memory usage rather than cpu usage. This will show you what processes are using the most memory. — theotherreceive, Dec 01 '11 at 16:35
Lessee...no swap...small RAM...all that pressure in the VM subsystem and no place to go...sounds familiar...hmm, it sounds similar (but not the same as) this? http://serverfault.com/questions/22217/linux-server-is-only-using-60-of-memory-then-swapping — Avery Payne, Dec 01 '11 at 21:48

thinice · Answer 1 · 2011-12-01T16:42:12.467

7

Dial down your Apache settings. With that level of memory something like:

StartServers 2
MinSpareServers 1
MaxSpareServers 3
MaxClients 10

(Make sure to restart apache)

Running a full LAMP stack on that much memory can be tough to eek by if you see even small amounts of requests.

edited Dec 01 '11 at 16:42

answered Dec 01 '11 at 16:35

thinice

4,716
21
38

score 5 · Accepted Answer · answered Dec 01 '11 at 17:48

5

You appear to have 1GB of RAM. This is the 21st century: to borrow from another user's quip book: My cellphone has more RAM than that! -- so suggestion #1 is add more RAM. 2GB is a bare minimum, and if you're running the full LAMP stack (Linux, Apache+PHP & a database like MySQL) on one machine you really need to go with 4GB.

You also appear to have no swap - This is Bad and Wrong, so suggestion #2 is "Configure some swap space".
Unix-Like systems, including Linux, expect to be able to swap when they run out of RAM. If they run out of RAM and there's no swap space to shove stuff into they usually have a fit and stop working.
This is probably what's happening to you.

If you're on some kind of virtual/cloud system and can't add swap space you're reduced back to suggestion #1 (add enough RAM so you don't run out).
If you configured a Linux installation with no swap space you can add a swap file (here's a howto, there are others if you google around), but you're probably better off reinstalling the machine and creating an actual swap partition it's more efficient.

answered Dec 01 '11 at 17:48

voretaq7

79,879
17
130
214

Touche. I always add swap to my own servers, but this one is hosted, so I'll have to look into adding more ram. – Jacksonkr Dec 01 '11 at 17:52
Uh, enabling swap leads to a whole new package of performance problems. You're running a full LAMP stack on 1G of RAM, you need to tone down how many apache process you're allowing – thinice Dec 01 '11 at 17:57
@rovangju Enabling swap leads to your system *not falling over when you run out of RAM*. Lousy performance can be tolerated to some extent,the entire site going down is usually considered to be unacceptable. Adding more RAM will give better performance (obviously), but only until you saturate your RAM, at which point you'll crash again. – voretaq7 Dec 01 '11 at 19:22
1

Is this a VPS? If it is, it looks like you may be on OpenVZ, which means you can't add swap, as that is handled by the host. – Rilindo Dec 01 '11 at 19:23
I'm not against swap, in this case I think it's ill advised as 'the solution' given the indicator that Apache is consuming 36% of the memory. – thinice Dec 01 '11 at 21:10
@rovangju Having swap available is "the solution" to `my system locks up when I run out of RAM!` (the primary problem). I'm not saying that this won't then expose a new problem (`my system is unbearably slow when I run out of RAM!`). It's a moot issue anyway though -- it sounds like this is a VPS that doesn't support Swap, so the only available option is throwing more (virtual) RAM at the problem . – voretaq7 Dec 01 '11 at 21:15

Ubuntu server using too much memory

top at 130%

free -m

top (pushed > once)

2 Answers2