how to execute a mysql query each time tail -f shows up a new line from a log file in linux

Question

I'm trying to monitor a log file with tail -f , parse it to extract some data with grep and pass the data as a mysql query. I could do that by passing each new line detected by tailf to a php script, but I don't know how to do that.. Or i could simulate tailf with php directly, but how to I monitor a file for changes with php? I think just by having a while , look for the size , remember the last position, seek it and read till the difference right? Anyone can give some hints on what's better to use? or simpler? Also i've heard named pipes could be a solution but don't know how to grab the data from there The logger is nginx by the way.. thank you

Answer 1

In shell you can do something like:

tail -f file.log|grep whatever-you-want|while read line; do 
  echo $line
done

Just change the line echo $line to what you want - you can call PHP or whatever you want in that line, $line is the the line from tail/grep combination.

Take a look at these SOq for more info:

• Use bash to read a file and then execute a command from the words extracted
• Bash script to read a file

Answer 2

you are mixing to different approaches. tail -f means you will have a process running all the time and if the log file is truncated (rotated) you will most probably get stuck. If you need good thing for log monitoring (i assume linux) then you have to monitor the file for changes and seek to the remembered position. The IMHO best way is to rely on an existing solution. syslog-ng seems to come firs to to mind. It can pipe your log files into your program. Though not the case with non sysloged logs. The I would opt for http://www.php.net/manual/en/intro.inotify.php. This will require a running process. As the final (the order depends on your situation) solution I would go for a periodic checking with cron.