Is there a way to join databases of two different data sources(ie. Mysql and Postgres SQL) using logstash and indexing it to elastic search?

Question

I am very new to ELK and want to know if there is a way around to join two databases from different sources (ie. MYSQL and Postgres) and indexing it to a single index in elasticsearch using logstash.

As I am able to achieve the same with the help of pyspark. But I want to achieve the same thing using log stash if it's possible!

Also, suggest some other feasible ways to achieve the same apart from the spark and logstash.

Thanks in Advance!

score 2 · Answer 1 · answered Nov 13 '19 at 07:32

2

You can definitely achieve this by sourcing data from one database using a jdbc input and then joining it with data coming from another database that is sourced with either the jdbc_static (if the data doesn't change too often) or jdbc_streaming (if the data changes more often) filters.

answered Nov 13 '19 at 07:32

Val

207,596
13
358
360

Also see this thread: https://stackoverflow.com/a/59047019/4604579 – Val Nov 26 '19 at 12:23
Hi @Val, Thanks for your response. i will try and come up with the queries if i face any trouble. – Shishir Dubey Dec 06 '19 at 09:16
Ok, feel free to come back in case you have any question – Val Dec 06 '19 at 09:17
Can i get some examples regarding this? – Shishir Dubey Dec 06 '19 at 10:04

Is there a way to join databases of two different data sources(ie. Mysql and Postgres SQL) using logstash and indexing it to elastic search?

1 Answers1

Linked