Piping tail output though grep twice

Question

Going with a typical Apache access log, you can run:

tail -f access_log | grep "127.0.0.1"

Which will only show you the logs (as they are created) for the specified IP address.

But why does this fail when you pipe it though grep a second time, to further limit the results?

For example, a simple exclude for ".css":

tail -f access_log | grep "127.0.0.1" | grep -v ".css"

won't show any output.

Answer 1

I believe the problem here is that the first grep is buffering the output which means the second grep won't see it until the buffer is flushed.

Try adding the --line-buffered option on your first grep:

tail -f access_log | grep --line-buffered "127.0.0.1" | grep -v ".css"

For more info, see "BashFAQ/009 -- What is buffering? Or, why does my command line produce no output: tail -f logfile | grep 'foo bar' | awk ..."

Answer 2

This is the result of buffering, it will eventually print when enough data is available.

Use the --line-buffered option as suggested by Shawn Chin or if stdbuf is available you can get the same effect with:

tail -f access_log | stdbuf -oL grep "127.0.0.1" | grep -v ".css"