0

I have a CentOS 7 VPS with Webmin and Virtualmin installed. I have several domains on this server as well as mail server(Postfix, Dovecot). Sometimes I can't access it (either from webmin using IP address or SSH) until I restart the machine from my provider panel. How can I detect the problem? What logs do I have to look?

Here is output for "grep -i 'killed process' /var/log/messages"

Oct  2 13:08:32 server kernel: Killed process 2840 (mysqld) total-vm:1641792kB, anon-rss:20564kB, file-rss:0kB
Oct  3 04:24:09 server kernel: Killed process 2874 (mysqld) total-vm:1637124kB, anon-rss:11628kB, file-rss:0kB
Oct  3 05:16:05 server kernel: Killed process 26933 (sh) total-vm:11636kB, anon-rss:4kB, file-rss:140kB
Oct  3 05:16:05 server kernel: Killed process 26884 (/usr/libexec/we) total-vm:165912kB, anon-rss:28kB, file-rss:132kB
Oct  3 05:16:05 server kernel: Killed process 26753 (/usr/libexec/we) total-vm:161776kB, anon-rss:80kB, file-rss:16kB
Oct  3 05:16:05 server kernel: Killed process 26852 (/usr/libexec/we) total-vm:132660kB, anon-rss:2476kB, file-rss:12kB
Oct  3 05:16:05 server kernel: Killed process 26903 (/usr/libexec/we) total-vm:132656kB, anon-rss:1964kB, file-rss:12kB
Oct  3 05:16:05 server kernel: Killed process 26715 (php-cgi) total-vm:486184kB, anon-rss:5772kB, file-rss:136kB
Oct  3 05:16:05 server kernel: Killed process 26953 (sh) total-vm:11636kB, anon-rss:0kB, file-rss:4kB
Oct  3 05:16:05 server kernel: Killed process 26749 (/usr/libexec/we) total-vm:113144kB, anon-rss:1444kB, file-rss:148kB
Oct  3 05:16:05 server kernel: Killed process 26954 (sh) total-vm:11636kB, anon-rss:4kB, file-rss:0kB
Oct  3 05:16:05 server kernel: Killed process 26795 (/usr/libexec/we) total-vm:113144kB, anon-rss:1132kB, file-rss:0kB
Oct  3 05:16:05 server kernel: Killed process 27303 (mysqld) total-vm:288396kB, anon-rss:9340kB, file-rss:0kB
Oct  3 05:16:05 server kernel: Killed process 27109 (php-cgi) total-vm:453836kB, anon-rss:2144kB, file-rss:20488kB
Oct  3 05:16:05 server kernel: Killed process 27111 (php-cgi) total-vm:453836kB, anon-rss:2308kB, file-rss:20000kB
Oct  3 05:16:05 server kernel: Killed process 26851 (/usr/libexec/we) total-vm:105188kB, anon-rss:2620kB, file-rss:4kB
Oct  3 05:16:05 server kernel: Killed process 26898 (/usr/libexec/we) total-vm:105188kB, anon-rss:2660kB, file-rss:44kB
Oct  3 05:34:26 server kernel: Killed process 27190 (php-cgi) total-vm:453836kB, anon-rss:2160kB, file-rss:20624kB
Oct  3 05:57:53 server kernel: Killed process 27186 (php-cgi) total-vm:453836kB, anon-rss:2324kB, file-rss:20280kB
Oct  3 06:00:46 server kernel: Killed process 27180 (php-cgi) total-vm:453836kB, anon-rss:2520kB, file-rss:20312kB
Oct  3 06:02:00 server kernel: Killed process 27187 (php-cgi) total-vm:453836kB, anon-rss:2240kB, file-rss:20076kB
Oct  3 06:04:16 server kernel: Killed process 2889 (named) total-vm:245060kB, anon-rss:512kB, file-rss:0kB
Oct  3 06:11:19 server kernel: Killed process 26709 (php-cgi) total-vm:471000kB, anon-rss:2228kB, file-rss:380kB
Oct  3 06:12:28 server kernel: Killed process 27308 (/usr/libexec/we) total-vm:96044kB, anon-rss:6292kB, file-rss:528kB
Oct  3 06:12:48 server kernel: Killed process 26788 (php-cgi) total-vm:464152kB, anon-rss:1852kB, file-rss:908kB
Oct  3 06:57:17 server kernel: Killed process 27465 (/usr/libexec/we) total-vm:102616kB, anon-rss:5608kB, file-rss:360kB
Oct  3 07:32:00 server kernel: Killed process 27457 (/usr/libexec/we) total-vm:102620kB, anon-rss:5080kB, file-rss:380kB
Oct  3 07:55:06 server kernel: Killed process 27474 (/usr/libexec/we) total-vm:102616kB, anon-rss:5156kB, file-rss:180kB
Rouhollah Mazarei
  • 111
  • 5

3 Answers3

0

First check the memory occupation. There can be some of your services leaking, so the memory consumption may rise over the time.

If you don't have sar in place, adjust it. There is quite nice howto:

http://www.thegeekstuff.com/2011/03/sar-examples/?utm_source=feedburner

Jaroslav Kucera
  • 1,545
  • 11
  • 18
0

It seems you have memory leak. The first of all check you logs.

grep -i 'killed process' /var/log/messages

Then you can try this trick to find out what proccess eats you memory. Add this command as crontab task:

ps aux | awk '{mem[$11]+=int($6/1024)}; {cpuper[$11]+=$3};{memper[$11]+=$4}; END {for (i in mem) {print cpuper[i]"% ",memper[i]"% ",mem[i]" MB ",i}}' | sort -k3nr | head -n 5 >> /tmp/memoryleak.log

Now you can check /var/log/memoryleak.log

Alexander Makarenko
  • 693
  • 4
  • 10
0

In addition to sar you can install atop and configure logs for it to find exact process responsible for high CPU/RAM usage.

antrost
  • 473
  • 3
  • 8