Since a few days, I see this kind of messages in my syslog:
Sep 23 14:28:42 server kernel: [138926.637593] php5-fpm[6455]: segfault at 7f9ade735018 ip 00007f9ae4026772 sp 00007ffd69b4fad0 error 4 in libmysqlclient.so.18.0.0[7f9ae3ff9000+2f1000]
Sep 23 14:28:44 server kernel: [138928.314016] php5-fpm[22742]: segfault at 7f9ade3db018 ip 00007f9ae4026772 sp 00007ffd69b4fad0 error 4 in libmysqlclient.so.18.0.0[7f9ae3ff9000+2f1000]
Sep 23 14:32:11 server kernel: [139135.318287] php5-fpm[16887]: segfault at 7f9ade4b3018 ip 00007f9ae4026772 sp 00007ffd69b4fad0 error 4 in libmysqlclient.so.18.0.0[7f9ae3ff9000+2f1000]
Sep 23 14:32:49 server kernel: [139173.050377] php5-fpm[668]: segfault at 7f9ade61a018 ip 00007f9ae4026772 sp 00007ffd69b4fad0 error 4 in libmysqlclient.so.18.0.0[7f9ae3ff9000+2f1000]
Sep 23 14:33:19 server kernel: [139203.396935] php5-fpm[26277]: segfault at 7f9ade6c0018 ip 00007f9ae4026772 sp 00007ffd69b4fad0 error 4 in libmysqlclient.so.18.0.0[7f9ae3ff9000+2f1000]
Sep 23 14:35:06 server kernel: [139310.048740] php5-fpm[27017]: segfault at 7f9ade46c018 ip 00007f9ae4026772 sp 00007ffd69b4fad0 error 4 in libmysqlclient.so.18.0.0[7f9ae3ff9000+2f1000]
Sep 23 14:35:19 server kernel: [139323.494188] php5-fpm[31263]: segfault at 7f9ade5e2018 ip 00007f9ae4026772 sp 00007ffd69b4fad0 error 4 in libmysqlclient.so.18.0.0[7f9ae3ff9000+2f1000]
Sep 23 14:36:10 server kernel: [139374.904308] php5-fpm[26422]: segfault at 7f9ade6cf018 ip 00007f9ae4026772 sp 00007ffd69b4fad0 error 4 in libmysqlclient.so.18.0.0[7f9ae3ff9000+2f1000]
Sep 23 14:37:25 server kernel: [139449.360384] php5-fpm[20806]: segfault at 7f9ade644018 ip 00007f9ae4026772 sp 00007ffd69b4fad0 error 4 in libmysqlclient.so.18.0.0[7f9ae3ff9000+2f1000]
I'm using debian 8 and MariaDB. In the beginning it was only once every 2 or 3 hours, but now it's several times per hours. After some research, I understand as it should be an memory problem, but I didn't find any solution to solve it.
This is what I see in mysqltuner:
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +ARCHIVE +Aria +BLACKHOLE +CSV +FEDERATED +InnoDB +MRG_MyISAM
[--] Data in InnoDB tables: 2G (Tables: 79)
[--] Data in MyISAM tables: 96M (Tables: 146)
[--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 52)
[!!] Total fragmented tables: 34
-------- Security Recommendations -------------------------------------------
[OK] All database users have passwords assigned
-------- Performance Metrics -------------------------------------------------
[--] Up for: 1d 16h 44m 38s (73M q [502.853 qps], 196K conn, TX: 572B, RX: 14B)
[--] Reads / Writes: 97% / 3%
[--] Total buffers: 17.3G global + 56.2M per thread (500 max threads)
[!!] Maximum possible memory usage: 44.8G (142% of installed RAM)
[OK] Slow queries: 0% (2K/73M)
[OK] Highest usage of available connections: 28% (141/500)
[OK] Key buffer size / total MyISAM indexes: 1.0G/32.6M
[OK] Key buffer hit rate: 100.0% (132M cached / 53K reads)
[OK] Query cache efficiency: 44.9% (50M cached / 113M selects)
[!!] Query cache prunes per day: 260596
[OK] Sorts requiring temporary tables: 0% (2K temp sorts / 2M sorts)
[OK] Temporary tables created on disk: 21% (6K on disk / 28K total)
[OK] Thread cache hit rate: 99% (141 created / 196K connections)
[OK] Table cache hit rate: 72% (500 open / 692 opened)
[OK] Open file limit used: 17% (429/2K)
[OK] Table locks acquired immediately: 99% (25M immediate / 25M locks)
[OK] InnoDB buffer pool / data size: 16.0G/2.4G
[!!] InnoDB log waits: 30
So the maximum memory to use is too high, but I ajusted my innodb buffer pool size to 16Go, for 32Go RAM that should be ok, I don't know what to do for optimize this.
The thing is, my memory general usage in the server is always under 89% (plus for caching). MySQL is actually using 50,6% of RAM. I don't know if there is link between all of this, but I prefere to put it here. Otherwise, everything seems to be ok in MySQL side...
Finally this the principal variables in my.cnf I adjusted which could have an effect on this:
max_connections = 100
max_heap_table_size = 64M
read_buffer_size = 4M
read_rnd_buffer_size = 32M
sort_buffer_size = 8M
query_cache_size = 256M
query_cache_limit = 4M
query_cache_type = 1
query_cache_strip_comments =1
thread_stack = 192K
transaction_isolation = READ-COMMITTED
tmp_table_size = 64M
nnodb_additional_mem_pool_size = 16M
innodb_buffer_pool_size = 16G
thread_cache_size = 4M
max_connections = 500
join_buffer_size = 12M
interactive_timeout = 30
wait_timeout = 30
open_files_limit = 800
innodb_file_per_table
key_buffer_size = 1G
table_open_cache = 500
innodb_log_file_size = 256M
Two days ago the server crashed for no reason in syslog execpt the segfault. Can segfault crash the system ? Any Idea for the segfault reason ? A few ways to understand the origin of the problem ?
