4

I'm using Groundwork / Nagios, and I'm trying to set up check_by_ssh. Right now, the command is:

$USER1$/check_by_ssh -i ~nagios/.ssh/id_dsa -H $HOSTADDRESS$ -t 60 -l "$USER24$" -C "/tmp/test"

where /tmp/test is a Hello World program.

but it's returning the message "Remote command execution failed:********************************************"

I have ssh keys set up for nagios to log into $HOSTADDRESS$ as $USER24$, but I'm still getting the error. (The private key is in ~nagios/.ssh on the groundwork box, and the public key is in ~/$USER24$/.ssh on the remote host)

So basically, check_by_ssh is failing to run any program.

fancyPants
  • 50,732
  • 33
  • 89
  • 96
Connor
  • 171
  • 1
  • 1
  • 8

8 Answers8

13

For some reason, adding the "-E" flag fixed it. According to the check_by_ssh man page, this is the ignore STDERR flag. Now I get the output from /tmp/test.

Final command:

$USER1$/check_by_ssh -i ~nagios/.ssh/id_dsa -H $HOSTADDRESS$

-t 60 -l "$USER24$" -C "/tmp/test" -E

Final output:

Hello World

Connor
  • 171
  • 1
  • 1
  • 8
  • This was what worked for me as well; in my specific case I'm wrapping `check_by_ssh` with `sshpass` (an unfortunate necessity due to hosting not allowing keyfiles for SSH access) – STW Jan 23 '14 at 20:18
9

If the check_by_ssh is failing because you are being required to verify the key authenticity, you can disable strict host key checking in your check_by_ssh options just as you can with the ssh client. This is a small security sacrifice, but if you are on a trusted private network, the trade off is negligible and you never have to verify that you wish to continue connecting, even on the first try:

/usr/lib/nagios/plugins/check_by_ssh -l nagios -o StrictHostKeyChecking=no
Brian Mains
  • 50,520
  • 35
  • 148
  • 257
Dave Stern
  • 511
  • 2
  • 7
  • 13
  • This obviously reduces your security a little bit, the better option might just be to clean out your known_hosts file on the icinga / nagios server – Martin W Feb 19 '20 at 11:04
4

In my case, the error was due to the update of the public key of the ssh service reached by nagios.

On the monitoring machine where nagios is installed, update or remove the file "/var/spool/nagios/.ssh/known_hosts" to remove all the public key.

and try again the check_by_ssh.

Example :

# ./check_by_ssh -H target_machine -C "/bin/ls" 
Remote command execution failed: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

# rm /var/spool/nagios/.ssh/known_hosts

#  ./check_by_ssh -H target_machine -C "/bin/ls /" 
bin
boot
cgroup
core
data
dev
etc
home
...
Cyrus31
  • 41
  • 4
4

Just ran into this on some of my systems. I couldn't figure it out but the -E flag did help. The reason this was occurring on my hosts is that I enabled an SSH banner to display the standard "Unauthorized use is prohibited by law blah-blah-blah". That banner is displayed via stderr, and so each of check_by_ssh calls failed with "Remote command execution failed".

So if host keys aren't your issue and you're not happy with -E, drop your banner. If you use a consistent username for your Nagios checks, you can exclude the nagios user from displaying the banner by using the Match option in sshd_config:

https://unix.stackexchange.com/questions/96975/disable-ssh-banner-for-specific-users-or-ips

Community
  • 1
  • 1
modea
  • 51
  • 1
3

You have to manually run the command the first time because it asks a question that you have to answer yes to. After that it no longer requires interaction.

You have to do that for each server you connect to.

fred
  • 39
  • 1
  • This answer is correct. You don't need to specify the key if you know which user is running the command, and you have your key in the default location (eg. if nagios is the user and you use an rsa key, ~nagios/.ssh/id_rsa). – Peter Oct 04 '12 at 09:10
1

I have found the answer.

In order to skip the banner add the file "config" on your .ssh directory on the nagios server

cat config LogLevel=QUIET

chmod 600 config

after that the banner will be skipped and nagios check will work without the -E flag

Regards Bogdan

Bogdan
  • 11
  • 1
0

In my case this happened after one of our servers got rebuilt. The problem was that the known_hosts file on the icinga master still had the SSH key of the target host from before it was rebuilt. The solution was to simply edit the known_hosts file and delete the line related to the target host. For me this was located in: /var/spool/icinga2/.ssh/known_hosts

Once you do this and run the check, the first time you will probably see a message like:

Remote command execution failed: Warning: Permanently added '172.16.0.90' (ED25519) to the list of known hosts.

That's normal, run the check again and it should be working

Martin W
  • 372
  • 1
  • 2
  • 14
0

If you have setup the public-key then you have to pass the key in your SSH command like: 

command_line    $USER1$/check_by_ssh -i /usr/local/nagios/etc/keys/$HOSTNAME$ -H $HOSTADDRESS$ -t 60 -l "$USER24$" -C "cd tmp"

Make sure the public-key has a permission 0600.

Rakesh Sankar
  • 9,337
  • 4
  • 41
  • 66