2

I am using a named pipe to trap syslog messages. I can then easily view syslog by doing something like

cat /var/log/local3.pipe | grep somefilter
or
grep somefilter /var/log/local3.pipe

These both output the syslogs to the console very nicely. However, if I then want to capture that to a file I get nothing, eg

cat /var/log/local3.pipe | grep somefilter >> somefile.log
or
grep somefilter /var/log/local3.pipe >> somefile.log

The file always remains as zero bytes. Does anyone know why? I'm using Red Hat Enterprise Linux 5. Thanks.

Additional info: For anyone who wants to reproduce this here's the full list of commands

su
<enter root password>
mkfifo /var/log/local3.pipe
chmod 644 /var/log/local3.pipe
echo "local3.* |/var/log/local3.pipe" >> /etc/syslog.conf
/etc/init.d/syslog restart
exit

then with one ssh session:

cat /var/log/local3.pipe

and in a second ssh session ("Test it" should show in first ssh session

logger -p local3.info "Test it"

then in the first session change it to

cat /var/log/local3.pipe >> somefile.log

send some more logs to local 3 (message needs to be different). Confirm that messages are going into somefile.log

logger -p local3.info "Test it 2"

then in the first session change it to

cat /var/log/local3.pipe | grep -i test >> somefile.log

now confirm that logs are not going to somefile.log

Note that the message needs to be different from the last message otherwise the logger doesn't send it immediately.

MikeKulls
  • 873
  • 1
  • 10
  • 22
  • Are you first doing `grep somefilter /var/log/local3.pipe` and only then `grep somefilter /var/log/local3.pipe >>somefile.log`? Because the contents of the pipe will have been consumed by then. – Cairnarvon Feb 20 '13 at 00:36
  • Hi Cairnarvon. Before executing the second command I always terminate the first. I can switch back and forward between these 2 commands and the first always works and the second always results in a zero byte file. If I do cat /var/log/local3.pipe >> somefile.log then that works. It appears that putting grep into the mix (and piping to a file) stops things working. – MikeKulls Feb 20 '13 at 00:53

1 Answers1

3

I can reproduce your problem, and this fixes it for me:

cat /var/log/local3.pipe | grep -i --line-buffered test >> somefile.log

I think the reason is that when standard output refers to a terminal (according to isatty(3)), then line buffering is used, but as soon as you redirect to a file, libc switches to normal fixed-size buffering. So you must have examined somefile.log before the grep process's standard output buffer was filled, therefore before it had any reason to write to the file.

Another way of forcing line-buffering is as follows:

cat /var/log/local3.pipe | stdbuf -oL grep -i test >> somefile.log

You can read more about all of this at these two great articles:

(I'm amused to notice that stdbuf was written by a guy who emailed me today about something completely different - small world!)

Adam Spiers
  • 17,397
  • 5
  • 46
  • 65