0

Server: Linode Dedicated 4G Ubuntu 18.04, Webmin + Virtualmin

Every now and then, mysql server stops and fails on restarting. Here is the logs I was able to find:

Apr 11 20:54:57 bamboo kernel: [ 2082.915857] [ 7664]   112  7664   406072    46295   745472        3             0 mysqld
Apr 11 20:54:57 bamboo kernel: [ 2082.915859] [ 7701]     0  7701   129687      307   679936     1435             0 apache2
Apr 11 20:54:57 bamboo kernel: [ 2082.915860] [ 7702]     0  7702   129683      253   667648     1487             0 apache2
Apr 11 20:54:57 bamboo kernel: [ 2082.915861] Out of memory: Kill process 7664 (mysqld) score 40 or sacrifice child
Apr 11 20:54:57 bamboo kernel: [ 2082.919284] Killed process 7664 (mysqld) total-vm:1624288kB, anon-rss:185180kB, file-rss:0kB, shmem-rss:0kB
Apr 11 20:54:57 bamboo kernel: [ 2082.941447] oom_reaper: reaped process 7664 (mysqld), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Apr 11 20:54:57 bamboo systemd[1]: mysql.service: Main process exited, code=killed, status=9/KILL
Apr 11 20:54:57 bamboo systemd[1]: mysql.service: Failed with result 'signal'.
Apr 11 20:54:57 bamboo systemd[1]: mysql.service: Service hold-off time over, scheduling restart.
Apr 11 20:54:57 bamboo systemd[1]: mysql.service: Scheduled restart job, restart counter is at 4.
Apr 11 20:54:57 bamboo systemd[1]: Stopped MySQL Community Server.

..........

Apr 11 14:02:53 bamboo systemd[1]: mysql.service: Scheduled restart job, restart counter is at 4.
Apr 11 14:02:53 bamboo systemd[1]: Stopped MySQL Community Server.
Apr 11 14:02:53 bamboo systemd[1]: Starting MySQL Community Server...
Apr 11 14:02:53 bamboo mysqld[4345]: Initialization of mysqld failed: 0
Apr 11 14:02:53 bamboo systemd[1]: mysql.service: Control process exited, code=exited status=1
Apr 11 14:02:53 bamboo systemd[1]: mysql.service: Failed with result 'exit-code'.
Apr 11 14:02:53 bamboo systemd[1]: Failed to start MySQL Community Server.
Apr 11 14:02:53 bamboo systemd[1]: mysql.service: Service hold-off time over, scheduling restart.

Looks like it is a memory issue. But I am hosting some very small wordpress sites on this server, and it is impossible for them to use up the memory.

The server graphs: enter image description here

You can see there is a CPU usage spike, and it is when the mysql databases got stopped.

I was not doing anything special at that time so it is quite hard for me to reproduce the problem and observer the possible cause. I guess to investigate this I need to look deeper into the logs, but this is totally beyond my knowledge.

Can someone shed some light regarding how to find the possible cause?

shenkwen
  • 199
  • 2
  • 14

1 Answers1

2

Out of memory: Kill process and oom_reaper... It's clearly a memory problem.

With MySQL stopped, what is the result of : free -h ?

Try to restart MySQL and check htop in an other console, to have a real time monitoring for processes, RAM, CPU...

Check the INNODB buffer size in a MySQL console, maybe it's more than your physical available memory :

show global variables like 'innodb_buffer_pool_size';

Do you have some SWAP partition ?

For the Wordpress side, there are Wordpress website, and Wordpress website... Do you have some plugins on it ? Some can be database intensive (or just not optimized).

1F987
  • 771
  • 1
  • 8