3

I try to execute etherwake based on a MQTT topic. The output of mosquitto_sub stops if I pipe it in a while statement.

works:

# mosquitto_sub -L mqtt://... | grep -o -E '([[:xdigit:]]{2}:){5}[[:xdigit:]]{2}'
00:00:00:00:de:ad
00:00:00:00:be:ef
00:00:00:00:ca:fe
(goes on and on)

does not work:

mosquitto_sub -L mqtt://... \
  | grep -o -E '([[:xdigit:]]{2}:){5}[[:xdigit:]]{2}' \
  | hexdump

Output stops after a single line:

0000000 1234 5678 9abc def0 abcd cafe 3762 3a65

The big picture is this one:

mosquitto_sub -L mqtt://... \
  | grep -o -E '([[:xdigit:]]{2}:){5}[[:xdigit:]]{2}' \
  | while read macaddr; do
      echo "send WOL to " $macaddr;
      /usr/bin/etherwake -D -b "$macaddr" 2>&1;
    done

Usually I am fine with the Linux shell but this time it simply gets stuck after the first line.

My guess is there is some problem with stdin or stdout (is not read or full etc.) in some kind. But I am out ideas.

By the way its an OpenWRT shell so an ash and no bash.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
muebau
  • 152
  • 7
  • 1
    Have you looked into whether this is [a buffering problem](http://mywiki.wooledge.org/BashFAQ/009)? What's the data rate? Have you waited long enough for ~4kb or so of content to accumulate in `grep`'s stdout, or are you calling it indefinitely-hung as soon as it doesn't write the expected content to output? – Charles Duffy Sep 01 '19 at 16:08
  • 2
    ...if that *is* the problem, this question is probably best flagged as a duplicate to https://stackoverflow.com/questions/49704671/no-output-while-piping-tail-through-grep – Charles Duffy Sep 01 '19 at 16:10
  • 1
    BTW, see https://eklitzke.org/stdout-buffering for a good explanation of the "why" of this behavior. – Charles Duffy Sep 01 '19 at 16:11
  • thank you all a lot. I think you all hit the point. Unfortunately OpenWRT does not have an unbuffered version of 'grep' or 'sed'. – muebau Sep 01 '19 at 16:39
  • Note that `etherwake` is also reading the pipe that is intended to be read only by `read`. It *may* be ignoring that stream and not consuming any data, but do be sure you really need to do `while read macaddr; do ...; /usr/bin/etherwake -D -b "$macaddr" 2>&1 <&-; ...` and close stdin for etherwake (or redirect it from /dev/zero or /dev/null, or something) – William Pursell Sep 01 '19 at 17:09
  • Possible duplicate of [No output while piping tail through grep](https://stackoverflow.com/questions/49704671/no-output-while-piping-tail-through-grep) – Doug Richardson Sep 01 '19 at 18:03

1 Answers1

2

The problem is indeed the "buffering" of grep when used with pipes.

Usually the '--line-buffered' switch should be used to force grep to process the data line by line instead of buffer the data.

Because grep on OpenWRT (busybox) does not have this switch 'awk' is used:

mosquitto_sub -L mqtt://... \
  | awk '/([[:xdigit:]]{2}:){5}[[:xdigit:]]{2}/{ print $0 }' \
  | hexdump

If there is no busybox version of grep used the solution would be like:

mosquitto_sub -L mqtt://... \
  | grep -o --line-buffered -E '([[:xdigit:]]{2}:){5}[[:xdigit:]]{2}' \
  | hexdump

Thank you all a lot for your help.

muebau
  • 152
  • 7