I am confronted with the following problem:
2002: Connection refused (mysql)
Information about my cloudserver:
1 vCPU / 2 GB RAM / 20 GB NVMe SSD
NO SWAP in standard-image from Hetzner (Provider)
If I run out-of-memory, mysql-server goes down and all other services continue running like ususal. Here is the log from mysql-server:
mysql-mailcow_1 | 2019-11-02 4:08:15 0 [Note] mysqld (mysqld 10.3.18-MariaDB-1:10.3.18+maria~bionic) starting as process 1 ...
mysql-mailcow_1 | 2019-11-02 4:08:15 0 [Note] InnoDB: Using Linux native AIO
mysql-mailcow_1 | 2019-11-02 4:08:15 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mysql-mailcow_1 | 2019-11-02 4:08:15 0 [Note] InnoDB: Uses event mutexes
mysql-mailcow_1 | 2019-11-02 4:08:15 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mysql-mailcow_1 | 2019-11-02 4:08:15 0 [Note] InnoDB: Number of pools: 1
mysql-mailcow_1 | 2019-11-02 4:08:15 0 [Note] InnoDB: Using SSE2 crc32 instructions
mysql-mailcow_1 | 2019-11-02 4:08:15 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
mysql-mailcow_1 | 2019-11-02 4:08:15 0 [ERROR] InnoDB: mmap(137297920 bytes) failed; errno 12
mysql-mailcow_1 | 2019-11-02 4:08:15 0 [ERROR] InnoDB: Cannot allocate memory for the buffer pool
mysql-mailcow_1 | 2019-11-02 4:08:15 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
mysql-mailcow_1 | 2019-11-02 4:08:15 0 [Note] InnoDB: Starting shutdown...
mysql-mailcow_1 | double free or corruption (out)
mysql-mailcow_1 | 191102 4:08:15 [ERROR] mysqld got signal 6 ;
mysql-mailcow_1 | This could be because you hit a bug. It is also possible that this binary
mysql-mailcow_1 | or one of the libraries it was linked against is corrupt, improperly built,
mysql-mailcow_1 | or misconfigured. This error can also be caused by malfunctioning hardware.
mysql-mailcow_1 |
mysql-mailcow_1 | To report this bug, see https://mariadb.com/kb/en/reporting-bugs
mysql-mailcow_1 |
mysql-mailcow_1 | We will try our best to scrape up some info that will hopefully help
mysql-mailcow_1 | diagnose the problem, but since we have already crashed,
mysql-mailcow_1 | something is definitely wrong and this may fail.
mysql-mailcow_1 |
mysql-mailcow_1 | Server version: 10.3.18-MariaDB-1:10.3.18+maria~bionic
mysql-mailcow_1 | key_buffer_size=134217728
mysql-mailcow_1 | read_buffer_size=2097152
mysql-mailcow_1 | max_used_connections=0
mysql-mailcow_1 | max_threads=1502
mysql-mailcow_1 | thread_count=0
mysql-mailcow_1 | It is possible that mysqld could use up to
mysql-mailcow_1 | key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 9393155 K bytes of memory
mysql-mailcow_1 | Hope that's ok; if not, decrease some variables in the equation.
mysql-mailcow_1 |
mysql-mailcow_1 | Thread pointer: 0x0
mysql-mailcow_1 | Attempting backtrace. You can use the following information to find out
mysql-mailcow_1 | where mysqld died. If you see no messages after this, something went
mysql-mailcow_1 | terribly wrong...
mysql-mailcow_1 | stack_bottom = 0x0 thread_stack 0x49000
mysql-mailcow_1 | 2019-11-02 5:20:57 0 [Note] mysqld (mysqld 10.3.18-MariaDB-1:10.3.18+maria~bionic) starting as process 1 ...
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Using Linux native AIO
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Uses event mutexes
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Number of pools: 1
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Using SSE2 crc32 instructions
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Completed initialization of buffer pool
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=669440117
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: 128 out of 128 rollback segments are active.
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Waiting for purge to start
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: 10.3.18 started; log sequence number 669440126; transaction id 1163900
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] Recovering after a crash using tc.log
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] Starting crash recovery...
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] Crash recovery finished.
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] Server socket created on IP: '::'.
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Warning] 'proxies_priv' entry '@% root@d9d8b67422e4' ignored in --skip-name-resolve mode.
mysql-mailcow_1 | 2019-11-02 5:20:58 6 [Note] Event Scheduler: scheduler thread started with id 6
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] mysqld: ready for connections.
mysql-mailcow_1 | Version: '10.3.18-MariaDB-1:10.3.18+maria~bionic' socket: '/var/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
mysql-mailcow_1 | 2019-11-02 5:20:58 0 [Note] InnoDB: Buffer pool(s) load completed at 191102 5:20:58
What could that be? I might need or recommend to create a SWAPFILE of about 4 GB like in this tutorial.
For example, there is also an explanation that it is not advisable to create a SWAPFILE for cloud servers, could this solve my problem? Or what would you recommend?
Thank you in advance.
UPDATE:
I had some time to get a closer look. In the syslog I can see the following entry:
Nov 15 13:48:32 mx kernel: [1122631.811889] Out of memory: Kill process 9712 (mysqld) score 72 or sacrifice child
Nov 15 13:48:32 mx kernel: [1122631.819983] Killed process 9712 (mysqld) total-vm:1512916kB, anon-rss:144324kB, file-rss:0kB
This is my my.cnf:
[mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
#innodb_file_per_table = TRUE
#innodb_file_format = barracuda
#innodb_large_prefix = TRUE
#sql_mode=IGNORE_SPACE,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
max_allowed_packet = 192M
max-connections = 500
innodb-strict-mode = 0
innodb_buffer_pool_size = 128M
key_buffer_size = 24M
thread_cache_size = 8
query_cache_type = 0
query_cache_size = 0
sort_buffer_size = 16M
read_rnd_buffer_size = 2M
tmp_table_size = 48M
max_heap_table_size = 48M
thread_stack = 128K
skip-host-cache
skip-name-resolve
log-warnings = 0
event_scheduler = 1
[client]
default-character-set = utf8mb4
[mysql]
default-character-set = utf8mb4
What can be adjusted here to avoid these problems?
