Consul script handler stdout not showing up in logs as documented

Question

I'm writting a script to be used as a handler in a watcher config in consul.

The documentation for script type hanlders states:

An executable handler reads the JSON invocation info from stdin. [...] Anything written to stdout is logged.

Based on that, I configured a script handler to watch for a k/v event:

  "watches": [
    {
      "type": "key",
      "key": "some_key",
      "handler_type": "script",
      "handler": "/opt/consul/script/key_handler.sh"
    }
  ]

And created the script accordingly

#!/bin/bash
read event_payload
echo "The value of the key is $(echo $event_payload | jq -r .Value | base64 -d)"

I changed the value of some_key in the consul k/v store and expected to see the stdout output somewhere in the logs. Consul is configured to log in a file and in syslog with a level of info. Here is the relevant part of the config file:

{
    "log_level": "INFO",
    "enable_syslog": true,
    "log_file": "/var/log/consul/",
    "log_rotate_duration": "24h",
    "log_rotate_max_files": 7
}

I looked in both syslog (i.e. journalct -u consul -f) and consul current log file (i.e. tail -f $(ls -trd /var/log/consul/* | tail -1). Both log facilities are ok and showing current consul activity. But I cannot find any trace of my script output.

If I modify the script and redirect the output to an arbitrary file, I can find the output I'm expecting so the script definitely does its job.

Did I miss anything in the configuration that is preventing these logs to show up ?

Zeitounator · Accepted Answer · 2020-10-07T13:52:42.577

I actually found the answer while I was writing this question and double checking my tests before posting. So I'll just share not to feel I wasted my time.

Watch script handler output is categorized as debug level by consul. I did not find any doc on that (although I might not have tried hard enough...), I inferred that from experiencing with the setting. You need to change the log_level to actually see the output.

{
    "log_level": "DEBUG",
}

An alternative if you don't want to change the global log_level is to use consul monitor:

# Set a token with needed access level if you use ACLs
# export CONSUL_HTTP_TOKEN="<your_token>"
consul monitor --log-level=DEBUG

This will let you explore the agents logs live with the desired level of output.

Ref: https://www.consul.io/docs/agent/options.html#_log_level

Consul script handler stdout not showing up in logs as documented

1 Answers1