0

I want to write a tool which parses log files at realtime. Log file can be written very fast, for example ~10k lines per second. I just need to get each line, then execute one regexp and write result to DB. Due to speed, it couldn't be done by reading file entirely, so I need to write alternative to something like tail -f. Tell me, please, how, and using what technology can I do this?

P.S. Sorry for bad English.

kopaty4
  • 2,236
  • 4
  • 26
  • 39

1 Answers1

0

You can store your logs in RAM-partition, with log-rotate limited to 512MB. So you'll be limited by the capacity of RAM with its high speed.

If your specify what application produces your logs, I can tell more. Have you tried for your tasks http://goaccess.io/?

Andrewus
  • 248
  • 1
  • 9