I have a little pickle here that I cannot seem to find a solution for;
As of a few days ago, my backups (VeeamB&R-9.5) started failing with the following error:
Veeam output;
Error: Cannot authenticate user. Soap fault. Cannot complete login due to an incorrect user name or password.Detail: 'InvalidLoginFault xmlns="urn:internalvim25" xsi:type="InvalidLogin"/InvalidLoginFault', endpoint: '' SOAP connection is not available
And also;
Task failed. Error: Cannot complete login due to an incorrect user name or password.
Now, nothing has changed (credential wise or config) prior to these errors. When I try logging into vCenter, it does not accept my credentials. I need to reboot the vCenter server in order to be able to login. Once the vCenter server has rebooted, backups run without issues - until it hits certain jobs, and it is not always the same jobs. It then gets stuck in this stale error state again.
So, I then drilled into Event Viewer of both vCenter and Veeam server and found this;
From the vCenter server:
The description for Event ID 4227 from source Tcpip cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
The handle is invalid
And;
The description for Event ID 113 from source MSiSCSI cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
0xefff0003 *172.20.20.5 0003260 ROOT\ISCSIPRT\0000_0
The handle is invalid
Also;
The description for Event ID 70 from source iScsiPrt cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
\Device\RaidPort1
The handle is invalid
on the Veeam server, I get;
The network interface "vmxnet3 Ethernet Adapter" has begun resetting. There will be a momentary disruption in network connectivity while the hardware resets.
Reason: The network driver detected that its hardware has stopped responding to commands. This network interface has reset 1 time(s) since it was last initialized.
I have tried updating VMTOOLS on the Veeam server - no effect.
I have drilled into the physical hosts diagnostics for the Veeam server - all good.
Also drilled into the SAN in question - all is green on that side as well.
The only thing that seems to fix the issue is a reboot of the vCenter server - but the issue just comes back after certain jobs are run.
Any pointers would be greatly appreciated!!
Thanks in advance.