28

I am using 'tail -f' to follow a log file as it's updated; next I pipe the output of that to grep to show only the lines containing a search term ("org.springframework" in this case); finally I'd like to make is piping the output from grep to a third command, 'cut':

tail -f logfile | grep org.springframework | cut -c 25-

The cut command would remove the first 25 characters of each line for me if it could get the input from grep! (It works as expected if I eliminate 'grep' from the chain.)

I'm using cygwin with bash.

Actual results: When I add the second pipe to connect to the 'cut' command, the result is that it hangs, as if it's waiting for input (in case you were wondering).

thecoshman
  • 8,394
  • 8
  • 55
  • 77
les2
  • 14,093
  • 16
  • 59
  • 76
  • If I stop after grep (without the pipe to 'cut'), it works (without stripping the first 24 or 25 chars). – les2 Jun 09 '09 at 20:55
  • The real problem here, is that `tail -f` never finishes, so the rest of the pipeline keeps waiting for more input – Hasturkun Jun 11 '09 at 00:24

3 Answers3

30

Assuming GNU grep, add --line-buffered to your command line, eg.

tail -f logfile | grep --line-buffered org.springframework | cut -c 25-

Edit:

I see grep buffering isn't the only problem here, as cut doesn't allow linewise buffering.

you might want to try replacing it with something you can control, such as sed:

tail -f logfile | sed -u -n -e '/org\.springframework/ s/\(.\{0,25\}\).*$/\1/p'

or awk

tail -f logfile | awk '/org\.springframework/ {print substr($0, 0, 25);fflush("")}'
Hasturkun
  • 35,395
  • 6
  • 71
  • 104
  • 1
    --line-buffered didn't work for me - that buffers the *output* of grep. If you do tail -f logfile|cut -c 25- it still just sits there. The buffering is being done on the output of tail. – Dennis Williamson Jun 10 '09 at 21:47
  • Revised, seems like you're being bitten by stdout buffering in cut, these should work for you – Hasturkun Jun 11 '09 at 00:23
  • Thank you! this is really useful when pipe-chaining multiple greps. e.g. someprogwithoutput | grep --line-buffered filterpattern | grep --color highlightpattern – Superole Mar 14 '12 at 10:35
  • Is that `fflush` a typo? – thecoshman Oct 19 '15 at 13:50
  • @thecoshman: I wrote this a while ago, but I don't think it's a typo. It's a forced output flush. the `""` argument is gawk specific though. – Hasturkun Oct 19 '15 at 13:56
12

On my system, about 8K was buffered before I got any output. This sequence worked to follow the file immediately:

tail -f logfile | while read line ; do echo "$line"| grep 'org.springframework'|cut -c 25- ; done
Dennis Williamson
  • 346,391
  • 90
  • 374
  • 439
-1

What you have should work fine -- that's the whole idea of pipelines. The only problem I see is that, in the version of cut I have (GNU coreutiles 6.10), you should use the syntax cut -c 25- (i.e. use a minus sign instead of a plus sign) to remove the first 24 characters.

You're also searching for different patterns in your two examples, in case that's relevant.

Adam Rosenfield
  • 390,455
  • 97
  • 512
  • 589