We have a galera cluster with four servers at two locations with the following setting:
server1 (location 1) with weight 2
server2 (location 1) with weight 2
server3 (location 2) with weight 2
server4 (location 2) with weight 1
Versions:
-----------
galera-25.3.25
10.2.22-MariaDB
wsrep_patch_version: wsrep_25.24
Everything is running fine except that every night at 5:30 (sometimes 5:31 or 5:32) all servers lose the connection between them. They regain it quickly - but I would like to understand what is happening and how to prevent it.
I already checked: there is no cronjob running at this time which could cause this and no other system shows any error.
The mysql error log shows warnings like:
WSREP: (75927761, 'ssl://0.0.0.0:4567') connection to peer e8e6dd8b with addr ssl://XXX.XXX.XXX.XXX:4567 timed out, no messages seen in PT3S
...
WSREP: discarding established (time wait) b69c4124 (ssl://XXX.XXX.XXX.XXX:4567)
...
WSREP: Quorum: No node with complete state
...
If you need more information, please let me know!