0

I've got a problem with a CentOS 7 dedicated server. I've had it for years, and never had any problems with it until now.

I did a yum update the other day, and then rebooted the machine, and it never came back up.

I'm using an IPMI session now (it's an HP ProLiant LG160 Gen8 box), and after going through all of the normal POST checks it gives me 4 boot options:

CentOS Linux (3.10.0.1062.18.1.el7.x86_64) 7 (Core)
CentOS Linux (3.10.0.1062.12.1.el7.x86_64) 7 (Core)
CentOS Linux 7 Rescue fc47167... [too lazy to type]
CentOS Linux (0-rescue-fc47167...) 7 (Core)

If I select the top option it goes to a black screen with a small smiley face icon top left, and a flashing cursor after it.

If instead I select the second option, I get the same but a few lines down in the center of the screen it says "c4".

Both screens seem to wait indefinitely for me to press a key, and then the screen clears and it goes back to

Press "F9" key for ROM-Based Setup Utility
Press "F10" key for Intelligent Provisioning
Press "F11" key for Default Boot Override Options
Press "F12" key for Network Boot

I press none of them and it clears again, and starts with

Attempting Boot from NIC

etc. But this time it appears to hang indefinitely rather than coming up with the four boot options.

I get left with this final line:

PXELINUX 6.03 PXE 2014-10-06 Copyright (C) 1994-2014 H. Peterr Anvin et al boot: _ <- flashing cursor

Any idea what's gone wrong? Is it a virus? Is it fixable? What's my next step?

Thanks!

Codemonkey
  • 1,086
  • 4
  • 19
  • 41
  • And if you try one of the rescue options? – Gerard H. Pille Apr 20 '20 at 11:50
  • I've not tried one yet, just copying some data off first. I'll report back. – Codemonkey Apr 20 '20 at 11:52
  • So, it's strange. If I issue a reboot and do nothing, the server goes through grub, selects the top item and then hangs on the smiley face as discussed previously. HOWEVER... if I hit F11, and manually pick item 5 ("One Time Boot to HDD"), I end up at the same grub screen, but this time, it'll autoselect the top item and boot without issue. No smiley face, no hanging, no problem. I've performed this multiple times now to test, and it seems to be 100% reproducible. Any idea why?! – Codemonkey Apr 20 '20 at 13:43
  • No, I wouldn't touch a system with IPMI with a stick. – Gerard H. Pille Apr 20 '20 at 13:59
  • Why not, out of interest? I know nothing about it and only ever use it when something's gone wrong, like now. – Codemonkey Apr 20 '20 at 14:28
  • You can't prevent people from getting access to your system. It's hard enough without IPMI. – Gerard H. Pille Apr 20 '20 at 14:36
  • People who work at my webhost certainly could, but I can't say that bothers me. Beyond that it's secure to outsiders (I think) as you have to log into my host's management console (with 2FA) to create the IPMI session before you can use it. – Codemonkey Apr 20 '20 at 15:56
  • I'm afraid even the IPMI's have their vulnerabilities, and they're not so easy to fix, if anyone bothers. – Gerard H. Pille Apr 20 '20 at 16:45

2 Answers2

1

My webhost eventually got back to me and advised me to clear the nvram:

WARNING: I don't know much about these commands other that, run at your own risk!

modprobe nvram
dd if=/dev/zero of=/dev/nvram

WARNING: I don't know much about these commands other that, run at your own risk!

They think they had a botched BIOS update, and sure enough that seems to have fixed things.

Codemonkey
  • 1,086
  • 4
  • 19
  • 41
0

Have you tried repairing the GRUB bootloader? I also recommend that you change the root user password by booting into single user mode.

Please refer to the documentation below. I hope this helps.

Basic System Recovery - https://docs.centos.org/en-US/centos/install-guide/Rescue_Mode/

srinivassc
  • 71
  • 3
  • Sorry, I should have updated the question. My webhost advised me to clear the nvram (`modprobe nvram`, `dd if=/dev/zero of=/dev/nvram`). They think they had a botched BIOS update, and sure enough that seems to have fixed things. – Codemonkey Apr 27 '20 at 06:56