3

I'm working on a medium-sized web application written in PHP that's running on a VPS with 512mb ram. The webapp hasn't been officially launched yet, so there isn't too much traffic going on, just me and a few other people working on it.

There is another slightly smaller webapp also hosted on this machine, among 4-5 other small static sites.

We are running Centos 5 32-bit & cPanel/WHM.

This is the result of running ps aux and, as you can see, it's not using 100% of the RAM. However, on the hypanel overview, it's always shown as using aroun 500MB ram, just for running apache, mysql, and the lowest-memory-footprint versions of the mail server, ftp server etc.

-bash-3.2# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   2156   664 ?        Ss   12:08   0:00 init [3]
root      1123  0.0  0.0   2260   548 ?        S<s  12:08   0:00 /sbin/udevd -d
root      1462  0.0  0.0   1812   568 ?        Ss   12:08   0:00 syslogd -m 0
named     1496  0.0  0.0   3808   820 ?        Ss   12:08   0:00 nsd
named     1497  0.0  0.0  10672   756 ?        S    12:08   0:00 nsd
named     1499  0.0  0.0   3880   584 ?        S    12:08   0:00 nsd
root      1514  0.0  0.1   7240  1064 ?        Ss   12:08   0:00 /usr/sbin/sshd
root      1522  0.0  0.0   2832   832 ?        Ss   12:08   0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
root      1534  0.0  0.1   3712  1328 ?        S    12:08   0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql -
mysql     1667  0.0  2.9 225680 30884 ?        Sl   12:08   0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql -
mailnull  1766  0.0  0.1   9352  1100 ?        Ss   12:08   0:00 /usr/sbin/exim -bd -q60m
root      1797  0.0  0.0   2156   708 ?        Ss   12:08   0:00 /usr/sbin/dovecot
root      1798  0.0  0.0   2632  1012 ?        S    12:08   0:00 dovecot-auth
root      1816  0.0  3.0  38580 32456 ?        Ss   12:08   0:01 /usr/local/bin/spamd -d --allowed-ips=127.0.0.1 --pidfi
root      1839  0.0  1.6  63200 17496 ?        Ss   12:08   0:00 /usr/local/apache/bin/httpd -k start -DSSL
root      1846  0.0  0.1   5416  1468 ?        Ss   12:08   0:00 pure-ftpd (SERVER)
root      1848  0.0  0.1   6212  1244 ?        S    12:08   0:00 /usr/sbin/pure-authd -s /var/run/ftpd.sock -r /usr/sbin
root      1856  0.0  0.1   4492  1112 ?        Ss   12:08   0:00 crond
root      1864  0.0  0.0   2356   428 ?        Ss   12:08   0:00 /usr/sbin/atd
dovecot   1927  0.0  0.1   5196  1952 ?        S    12:08   0:00 pop3-login
dovecot   1928  0.0  0.1   5196  1948 ?        S    12:08   0:00 pop3-login
dovecot   1929  0.0  0.1   5316  2012 ?        S    12:08   0:00 imap-login
dovecot   1930  0.0  0.2   5416  2228 ?        S    12:08   0:00 imap-login
root      1939  0.0  0.1   3936  1964 ?        S    12:08   0:00 cPhulkd - processor
root      1963  0.0  0.8  15876  8564 ?        S    12:08   0:00 cpsrvd (SSL) - waiting for connections
root      1966  0.0  0.7  15172  7748 ?        S    12:08   0:00 cpdavd - accepting connections on 2077 and 2078
root      1990  0.0  0.2   5008  3136 ?        S    12:08   0:00 queueprocd - wait to process a task
root      2017  0.0  2.9  38580 31020 ?        S    12:08   0:00 spamd child
root      2018  0.0  0.5   8904  5636 ?        S    12:08   0:00 /usr/bin/perl /usr/local/cpanel/bin/leechprotect
nobody    2021  0.0  3.2  66512 33724 ?        S    12:08   0:00 /usr/local/apache/bin/httpd -k start -DSSL
nobody    2022  0.0  3.1  67812 33024 ?        S    12:08   0:00 /usr/local/apache/bin/httpd -k start -DSSL
nobody    2024  0.0  1.9  64364 20680 ?        S    12:08   0:00 /usr/local/apache/bin/httpd -k start -DSSL
root      2027  0.0  0.4   9000  4540 ?        S    12:08   0:00 tailwatchd
root      2032  0.0  0.1   4176  1836 ?        SN   12:08   0:00 cpanellogd - sleeping for logs
nobody    3096  0.0  1.9  64572 20264 ?        S    12:09   0:00 /usr/local/apache/bin/httpd -k start -DSSL
nobody    3097  0.0  2.8  66008 30136 ?        S    12:09   0:00 /usr/local/apache/bin/httpd -k start -DSSL
nobody    3098  0.0  2.8  65704 29752 ?        S    12:09   0:00 /usr/local/apache/bin/httpd -k start -DSSL
nobody    3099  0.0  3.1  67260 32816 ?        S    12:09   0:00 /usr/local/apache/bin/httpd -k start -DSSL
andrei    3448  0.0  0.1   3204  1632 ?        S    12:50   0:00 imap
nobody    3537  0.0  1.9  64308 20108 ?        S    13:01   0:00 /usr/local/apache/bin/httpd -k start -DSSL
nobody    3614  0.0  1.9  64576 20628 ?        S    13:10   0:00 /usr/local/apache/bin/httpd -k start -DSSL
nobody    3615  0.0  1.3  63200 14672 ?        S    13:10   0:00 /usr/local/apache/bin/httpd -k start -DSSL
root      3626  0.0  0.2  10232  2964 ?        Rs   13:14   0:00 sshd: root@pts/0
root      3648  0.0  0.1   3844  1600 pts/0    Ss   13:14   0:00 -bash
root      3826  0.0  0.0   2532   908 pts/0    R+   13:21   0:00 ps aux

Lately, without any significant changes to the configuration, the memory usage started peaking and going over 512, causing the virtual server to kill apache, basically murdering our site in the process.

Here is the result of free -m:

-bash-3.2# free -m
             total       used       free     shared    buffers     cached
Mem:          1024        381        642          0          0          0
-/+ buffers/cache:        381        642
Swap:            0          0          0

Another notice:

Apache seems to get killed when usage reaches 400/512mb of ram. This didn't use to happen before. It's pretty strange.

Do you have any idea if this is normal and more resources should be acquired? I don't think so, since there isn't too much data or traffic online yet.

Edit 2:

[Sat Apr 07 18:04:21 2012] [notice] Graceful restart requested, doing restart
[Sat Apr 07 18:04:22 2012] [notice] seg fault or similar nasty error detected in the parent process
(after manual restart)
[Sat Apr 07 18:28:51 2012] [notice] suEXEC mechanism enabled (wrapper: /usr/local/apache/bin/suexec)
etc.

Here's the log from one of Apache's crashes. This is baffling.

Andrei Bârsan
  • 153
  • 2
  • 8

2 Answers2

0

free -m shows that there should be enough free RAM. When a process gets killed because of too few RAM being available, this is called an "oom kill"m where "oom" stands for out of memory.

You can even tell Linux which processes to kill in what order when this situation occurs.

Besides that, MySQL reserves a lot of RAM on our system, which is normal. You can decrease the value by editing the my.cnf file and tuning the various variables of the MySQL config file.

Furthermore you should put some efforts in optimizing Apache in order to keep the RAM consumption low. It heavily depends on your PHP application if Apache and PHP consume much RAM or not.

In addition you should create a SWAP file which helps you in case of your memory running full.

PythonLearner
  • 1,032
  • 2
  • 12
  • 31
  • Well, I've reduced the all the memory tweaks in my.cnf significantly, and after some googling it seems that for my case, even decreases from 64MB to 265k wouldn't impact mysql's performance too much. Now the memory usage is better, and I'm going to create a swap partition. The thing is, I'm actually on a OpenVZ VPS apparently, which is rather different from CentOS. It doesn't seem to allow users to create swap file. I've found a workaround, what do you think - would it be worth it? https://www.intovps.com/client/knowledgebase/4/How-to-enable-swap-inside-your-VPS.html – Andrei Bârsan Apr 07 '12 at 12:56
  • Uh, that's bad :\ I recommend talking to your provider that you need more RAM or SWAP. But the suggestion from the intovps forums only should be applied when having enough free RAM and the system not using swap at any point of time. I recommend not to use that script for your VPS. – PythonLearner Apr 07 '12 at 13:06
  • But *do* I need more RAM? It's just a light mysql server + php + email, why would it need more RAM? 512 seems decent for the system. Moreover it used to work 100% ok until a couple of days ago. And nothing was uploaded that might cause such problems. Just a cPanel update. Could that have anything to do with it? – Andrei Bârsan Apr 07 '12 at 13:09
  • Managed to get it down do about 250/512 by further lowering mysql's buffering, disabling spamd, limiting cpanels memory usage and setting the MaxSpareServers config variable to 5 in httpd.conf – Andrei Bârsan Apr 07 '12 at 13:52
  • That sounds great! Well, the cPanel upgrade could couse this, although I find it quite unusual. Asking the cPanel community could help! – PythonLearner Apr 07 '12 at 15:28
  • I've submitted a ticket to my hosting provider. They should be able to give me some pointers. I'll post more information here as soon as it becomes available. – Andrei Bârsan Apr 07 '12 at 15:44
  • The webhost told me to check apache's error logs, and so I did, only to find reports of a segfault. (question updated) – Andrei Bârsan Apr 08 '12 at 08:30
  • In my experience apache is a major resource hog. I have switched a lot my websites to nginx and resource usage is reduced. It can be reduced further by tuning the nginx equivalent of the MaxSpareServers and MinSpareServers in Apache. – vfclists Apr 08 '12 at 08:35
  • Segfaults are an indicator for memory mismanagement or bugs. Maybe updating to another version of Apache will help? Besides that, if switching to nginx (as vfclists suggests) is an option for you, this will reduce your RAM consumption. See my blog for a small tutorial: http://www.xenuser.org/2011/08/28/running-phpbb3-on-a-high-performance-monster-or-how-nginx-php-fpm-php-apc-postgres-will-serve-you-well/ – PythonLearner Apr 08 '12 at 09:49
  • Thanks for the tips. Yeah, I've been thinking about that, since I'm pretty positive Apache is overkill for what we're running. nginx is compatible with cPanel, right? – Andrei Bârsan Apr 08 '12 at 18:44
0

The apache log shows a graceful restart, which might be due to log rotation: is it the same time each day?

This restart then runs into a segfault or similar problem which may be due to a library mismatch. Was PHP installed independently of apache? Did this problem only start after the cpanel upgrade (which might have upgraded some library or other)?

Try stopping and starting apache: does this error always occur? What if you disable php or other modules?

You should consider making this a new question, if it is not related to the memory use that you first asked about.

ramruma
  • 2,740
  • 1
  • 15
  • 8
  • Hello, nope, I haven't seen any sort of pattern. The problem might have been a misconfigured PHP .so file, namely the intl extension. It hadn't been properly compiled and it was missing. That happened a lot of time before, but it seems it hasn't bitten me until now (the upgrade probably stirred it up). I've removed it from php's config and everything seems to be in order. Odd. – Andrei Bârsan Apr 08 '12 at 14:30