Stack Overflow
Stack Overflow

Questions tagged [logstash-jdbc]

187 questions
0
votes
1 answer

Logstash not populating ElasticSearch from MySQL

I'm currently in the early stages of setting up some Kibana dashboards using the ELK stack and a MySQL database. According to the logstash config check utility, I have a valid .conf file, but nothing is showing up in elastic. First off, my DB is…
tparrott
  • 343
  • 1
  • 4
  • 16
0
votes
1 answer

How to turn off pre-check of how many rows are in the resultset in logstash output

I'm trying to turn off the pre-select logstash does to determine the count of rows, but ExaSol DB does not support any limits in any aggregation, is there any way to turn it off in logstash? input { jdbc { jdbc_driver_library =>…
d0mik
  • 1
0
votes
2 answers

Logstash 2.3.4 How to load nested document in elasticsearch using logstash-jdbc plugin

I am currently using elasticsearch 2.3.4 and logstash 2.3.4 to load relational data from Oracle db into my elasticsearch index using logstash-jdbc plugin. As suggested in various posts, I am using aggregate filter for this. Still I am not able to…
Vinay Rawat
  • 9
  • 3
0
votes
1 answer

Logstash as a service with JDBC input

I currently have a service which runs several queries to build data in my database. It then uses the Logstash's JDBC input plugin to send data to Elastic Search. The problem is that Logstash takes quite a lot of time bootstrapping (sometimes more…
Lucat
  • 2,242
  • 1
  • 30
  • 41
0
votes
1 answer

How to dynamically define elasticsearch index with logstash?

See below a logstash config file to pull records from a mysql database into an elasticsearch index using the jdbc plugin. How can it be modified so that there are separate indexes produced based on a company_id value found in the database, like…
Streamline
  • 2,040
  • 4
  • 37
  • 56
0
votes
1 answer

Does Elasticsearch performs full table scan on my Oracle table everytime when Logstash is run?

I wanted to know if the Elasticsearch performs the full table scan on my Oracle table if I try to ingest that table's delta data using Logstash
Srinivasulu Chinna
  • 49
  • 5
0
votes
1 answer

logstash sum values from jdbc input

I'm new on logstash and i'm trying to sum the value from two colums from my database and generate a new metric. I've exhausted all my alternatives. this is my conf file. the new varible that I'm creating is the 'tod_ped' I created another variable…
Thiago
  • 1
  • 2
0
votes
1 answer

Add tags in Elasticsearch using logstash jdbc plugin

Suppose we have 2 table 1) Customer (CustID, Name) 2) Purchases (Cust_ID, Purchase_ID) There is one to many (1:n) relation between (1) & (2). Now using logstash I want to insert this data from sql into elasticsearch. The problem is if I take join…
Palash Tichkule
  • 296
  • 4
  • 11
0
votes
0 answers

Convert multi dimensional array in Logstash

How can i convert multi dimensional SQL json string to valid JSON object so that it can be mapped to Elasticsearch index template field types. Example case: Let's assume that this is my Elasticsearch index template. { "settings": { …
Sami Pylkkänen
  • 1
  • 1
0
votes
0 answers

LogStash: MongoDB input plugin to ElasticStash

Is there any plugin to input MongoDB using LogStash to store the data in ElasticSearch. There is no search plugins available in elasticsearch website. https://www.elastic.co/guide/en/logstash/current/input-plugins.html
iamjustcoder
  • 4,714
  • 10
  • 33
  • 46
0
votes
1 answer

Logstash convert the "yyyy-MM-dd" to "yyyy-MM-dd'T'HH:mm:ss.SSSZ"

I use the logstash-input-jdbc plugin to sync my data from mysql to elasiticsearch. However, when I looked at the data in elasticsearch, I found that the format of the fields of all date types changed from "yyyy-MM-dd" to…
wangjinhao
  • 123
  • 2
  • 9
0
votes
0 answers

How to merge multiple jdbc inputs in logstash configuration based on condition of single column?

I have two tables in sql server i.e. AppDetails and AppBranchDetails. I want to read all rows of these two tables and merge based on the condition. Below are the two queries which I want to run: select id as colg_id, name, sirname from AppDetails…
ketan
  • 2,732
  • 11
  • 34
  • 80
0
votes
1 answer

Install Logstash error Errno::ENOMEM Duplicate

i am trying to install elasticsearch with logstash for every mysql action. i stuck when trying to install logstash . heres my command. [root@centos-elasticsearch tampung]# rpm -ivh logstash-6.5.2.rpm Preparing... …
user2809386
0
votes
1 answer

How to use the field returned by sql for logstash if statement

Sql statement statement => "select ref_id,index_name,type,content from t_backup_es_data where backup_id >= :sql_last_value" The value of the returned type column has a proposal and quotation if[%{type}] == "quotation" { elasticsearch { …
Jacky0312
  • 28
  • 3
0
votes
1 answer

Can't import mysql data into elastic search

I'm trying to follow along with a course example haven't recieved any help in the FAQ's, tried everything I could find on google and here. I'm not using docker just running this demo on my local machine(Ubunutu 18.04), both elastic search and mysql…
Kelvin Baumgart
  • 151
  • 1
  • 2
  • 13
1 2 3
…
12 13