How to "tail -f" with "grep" save outputs to an another file which name is time varying?

Question

STEP1

Like I said in title, I would like to save output of tail -f example | grep "DESIRED" to different file

I have tried

tail -f example | grep "DESIRED" | tee -a different

tail -f example | grep "DESIRED" >> different

all of them not working

and I have searched similar questions and read several experts suggesting buffered but I cannot use it.....

Is there any other way I can do it?

STEP2

once above is done, I would like to make "different" (filename from above) to time varying. I want to keep change its name in every 30minutes.

For example like

20221203133000

20221203140000

20221203143000

...

I have tried

tail -f example | grep "DESIRED" | tee -a $(date +%Y%m%d%H)$([ $(date +%M) -lt 30 ] && echo 00 || echo 30)00

The problem is since I did not even solve first step, I could not test the second step. But I think this command will only create one file based on the time I run the command,,,, Could I please get some advice?

Answer 1

Below code should do what you want.

Some explanations: as you want bash to execute some "code" (in your case dumping to a different file name) you might need two things running in parallel: the tail + grep, and the code that would decide where to dump.

To connect the two processes I use a name fifo (created with mkfifo) in which what is written by tail + grep (using > tmp_fifo) is read in the while loop (using < tmp_fifo). Then once in a while loop, you are free to output to whatever file name you want.

Note: without line-buffered (like in your question) grep will work, will just wait until it has more data (prob 8k) to dump to the file. So if you do not have lots of data generated in "example" it will not dump it until it is enough.

rm -rf tmp_fifo
mkfifo tmp_fifo

(tail -f input | grep --line-buffered TEXT_TO_CHECK > tmp_fifo &)

while read LINE < tmp_fifo; do
  CURRENT_NAME=$(date +%Y%m%d%H)
  # or any other code that determines to what file to dump ...
  echo $LINE >> ${CURRENT_NAME}
done