0

I would want to process log files like CEP (Complex Event Processing), but CEP is intended to use streams, not log files as source.

Are there exist CEP like tools for log files (batch data with timestamps) source ?

Or should i convert my log files into streams before processing them by existing CEP tool (or framework) ?

I have tried FlinkCEP and it did the work by some way, but it does not seem to suitable for complex event patterns, or it is very hard to learn how to use it.. There are very few examples for complex event patterns..

But what about Siddhi, is it applicable for log files ?

BR Esa

Eras Rasmuson
  • 11
  • 1
  • 5

1 Answers1

0

Siddhi will work for log files, you can use siddhi as an embedded mode, or you can also use siddhi as a sever by using WSO2 Stream Processor.

In both cases use Siddhi file source, and use text mapping in the text mapping to achieve this. You can define beginning and ending regex patterns to identify each log messages and use regex to extract each element in logs for processing. This will help you to convert the logs as a stream and then you will be able to process the logs in a streaming mode.

suho
  • 912
  • 6
  • 12