We're running a LAMP stack VPS on CentOS 7 hosting a few websites, MariaDB databases, and associated services. In the middle of the night our server mysteriously went completely offline.
When we discovered the issue we power cycled the VPS and the server came back up - but I was greeted by an SSH warning that the RSA2 fingerprint had changed when I logged back in (which seems very suspicious). A parse of the logs seems to indicate that the eth1 connection suddenly stopped working:
Full log from /var/log/messages: http://pastebin.com/Gbmitkhs
Here are the last few lines before server went offline:
Dec 17 02:24:53 WebServer NetworkManager[487]: <warn> (eth1) firewall zone remove failed [102402]: (4) Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
Dec 17 02:25:30 WebServer systemd-logind: Failed to start user slice: Connection timed out
Dec 17 02:25:31 WebServer systemd-logind: Assertion 's->user->slice' failed at src/login/logind-session.c:510, function session_start_scope(). Aborting.
Dec 17 02:25:32 WebServer systemd: systemd-logind.service: main process exited, code=killed, status=6/ABRT
Dec 17 02:25:32 WebServer systemd: Unit systemd-logind.service entered failed state.
Dec 17 02:25:33 WebServer systemd: systemd-logind.service failed.
Dec 17 02:25:34 WebServer systemd: systemd-logind.service has no holdoff time, scheduling restart
I didn't notice any suspicious activity in a preliminary look at the security logs, or the Apache access logs (besides bot crawling activity).
What could have caused the failure, and subsequent change of the server's RSA2 fingerprint?
