I've created a Nagios check but it keeps returning NRPE:Unable to read output, any idea why?

Question

I created a Nagios check which checks our pacemaker using the crm_mon command. The check is configured in the same way on both Nagios server and client's nrpe.cfg: The command definition in nrpe.cfg looks like that:

[root@Nagios_clt plugins]# grep pacemaker /etc/nagios/nrpe.cfg 
command[check_pacemaker]=/usr/bin/sudo /usr/sbin/crm_mon -s

I did two tests: In the first one, I'm just using the line you see above and then from the Nagios server I get:

[root@Nagios_srv ]# /usr/lib64/nagios/plugins/check_nrpe -H 192.168.57.157 -c check_pacemaker
NRPE: Unable to read output
[root@Nagios_srv ]# /usr/lib64/nagios/plugins/check_nrpe -H 192.168.57.157
NRPE v2.14

In the second one, I wrote a different command definition:

[root@Nagios_srv ]# grep pacemaker /etc/nagios/nrpe.cfg 
command[check_pacemaker]=/usr/lib64/nagios/plugins/check_pacemaker.sh

While /usr/lib64/nagios/plugins/check_pacemaker.sh looks like that:

[root@Nagios_svr ]# cat /usr/lib64/nagios/plugins/check_pacemaker.sh
#!/bin/bash
/usr/bin/sudo /usr/sbin/crm_mon -s

I've chmod +x the check_pacemaker.sh file. None of these worked. If I run the check_pacemaker.sh file locally on the Nagios client, I get the correct result:

[root@Nagios_clt ]# /usr/lib64/nagios/plugins/check_pacemaker.sh
Ok: 2 nodes online, 10 resources configured

If I run the command locally using check_nrpe I get this result:

[root@Nagios_clt plugins]# /usr/lib64/nagios/plugins/check_nrpe -H localhost
NRPE v2.14
[root@Nagios_clt plugins]# /usr/lib64/nagios/plugins/check_nrpe -H localhost -c check_pacemaker
NRPE: Unable to read output

Some other stuff I've configured:

[root@Nagios_clt plugins]# grep Defaults /etc/sudoers
#Defaults    requiretty
[root@Nagios_clt plugins]# grep nagios /etc/sudoers
nagios  ALL=NOPASSWD:/usr/lib64/nagios/plugins/*

The check_command looks like that:

define command{
        command_name    check_pacemaker
        command_line    /usr/lib64/nagios/plugins/check_pacemaker.sh
        }
[root@Nagios_clt plugins]# service iptables status
iptables: Firewall is not running.

Other checks on this server are working using nrpe: Other checks on the same server And I don't understand why, does anyone have an idea?

My first guess is that the user who runs the NRPE daemon, often `nagios`, doesn't have sudo privileges to run `/usr/sbin/crm_mon`. I can see it has privileges to run stuff in the plugins directory, but your plugin isn't written that way - it invokes `crm_mon` via `sudo`. — MadHatter, Jan 29 '14 at 14:08
If you `su - nagios` and try to run the plugin, you'd probably see the behavior @MadHatter is pointing out. Give nagios sudo access to that `crm_mon` command too. — Nathan C, Jan 29 '14 at 14:20
I don't know why I didn't see it myself, but thanks a lot guys, it works, please create an answer so I can accept it. Thanks! — Itai Ganot, Jan 29 '14 at 14:27

MadHatter · Accepted Answer · 2014-01-29T17:15:53.143

Your issue is lack of clarity about who's running what with sudo. Your plugin calls crm_mon with sudo /usr/bin/crm_mon, but instead of giving the nagios user sudo privileges to run the crm_mon binary, it only has privileges to run plugins (ie, anything in /usr/lib64/nagios/plugins/).

Either add passwordless sudo privileges for the /usr/bin/crm_mon binary for the nagios user, or change the plugin invocation to use sudo:

define command{
        command_name    check_pacemaker
        command_line    sudo /usr/lib64/nagios/plugins/check_pacemaker.sh
        }

and remove the sudo from check_pacemaker.sh.

score 0 · Answer 2 · answered Jan 29 '14 at 17:56

When troubleshooting a command run as a regular user via nrpe, you can fully imitate the solution, sudo and all, with another user first. Stop testing it as root.

You might find there is an issue with the sudoers set up, or there may be file access issues on secondary files such as those under /etc or /var used by the shell script.

Whatever the problem is, you can see the actual error by setting up your non-root user (e.g. itai) to have the same sudoers rights and try /usr/lib64/nagios/plugins/check_pacemaker.sh as that user. nrpe does not pass back errors, so you'll never get clues that way.

I've created a Nagios check but it keeps returning NRPE:Unable to read output, any idea why?

2 Answers2