1

Recently I have used Jdbc_streaming filter plugin of logstash, it is very helpful plugin which allows me to connect with my database on the fly and perform checks against my events. But are there any drawbacks or pitfall of using this filter. I mean I have the following queries :

For example , I am firing select query against each of my events.

  • Is it a good idea to query my database for each event. I mean what if I am processing a syslog event of a server which is continuously sending me data, in that case for each event I will be triggering a select query on my database so how will my database will react in terms of load and response time.
  • What about the number of connections, how they are managed.
  • How this will behave if I join multiple tables.

I hope I am able to convey my question. I just want to understand , how exactly it is working in back end and does querying my database at massive speed will degrade my database performance.

Shivkumar Mallesappa
  • 2,875
  • 7
  • 41
  • 68

1 Answers1

0

I am not sure whether this answer is correct or not. But as per my experience , logstash works in sequential manner for the above plugin. It creates only single connection to RDS and query's the DB for each record. So there is no connection overhead, but then it degrades the performance by many folds. This answer is just from my experience, it might be possible that this can be a completely wrong answer. Any edits or answers are welcome.

Shivkumar Mallesappa
  • 2,875
  • 7
  • 41
  • 68