Questions tagged [logstash-jdbc]
187 questions
1
vote
1 answer
Logstash jdbc plugin issue
I am using logstash jdbc plugin to transfer data from db to elasticsearch.
When picking data from db the column names are in lowercase.
But I need the column names to be changed to uppercase and the values to be inserted in the already created index…
sri
- 331
- 1
- 4
- 11
0
votes
1 answer
Duplication in logstash pipeline (input elasticsearch and output sql database)
I am using elasicsearch index as my input in my logstash config and the output is jdbc-output plugin logstash that send logs to sql database table columns , and the problem is I have duplication in sql database , I used uuid filter plugin logtash…
Ali farahzadi
- 274
- 2
- 10
0
votes
0 answers
New JDBC index not appearing in Kibana's Discover data view
I added the below config to my conf.d/logstash-simple.conf file.
input {
beats {
jdbc {
add_field => { "[index_name]" => "pglogdb" }
add_field => { "[@metadata][beat]" => "jdbc" }
add_field => {…
runnerpaul
- 5,942
- 8
- 49
- 118
0
votes
0 answers
Java errors when running Logstash with database configuration
I added the below jdbc config to my conf.d/logstash-simple.conf file.
jdbc {
add_field => { "[index_name]" => "pglogdb" }
add_field => { "[@metadata][beat]" => "jdbc" }
add_field => { "[@metadata][version]" => "1.1" }
tags =>…
runnerpaul
- 5,942
- 8
- 49
- 118
0
votes
0 answers
Logstash is not pushing correct data to elastic while using aggregated
I am pushing data to elastic via logstash , In logstash I have aggerated the data as parent child.
This pushing data but missed some data while aggerate, for example I have one card and that have 10 chapters then it never push 10 chapters to…
Vijendra Singh
- 1
- 1
0
votes
1 answer
LogStash - Issue with sql_last_value and last_run_metadata_path
I'm new to ELK and I'm currently struggling with some setup - maybe I missed a point.
I have set up my Logstash to parse my DB (MySql), and I've got 2 cases:
"Unforeseen maintenance"
-> In this case, I would need to stop/start Logstash manually…
0
votes
0 answers
logstash as a service : issue writing last_run_metadata_path
The logstash service runs on an AWS ec2 instance, and also I have a pipe.conf file that has this config:
input {
jdbc {
jdbc_connection_string => "jdbc:sqlserver://xxxxx:xxxx;databaseName=xxx;encrypt=true;trustServerCertificate=true"
…
javier_orta
- 457
- 4
- 15
0
votes
1 answer
How to map array inside message in Logstash HTTP Output
I am using Logstash to update by query existing Elasticsearch documents with an additional field that contains aggregate values extracted from Potgresql table.
I use elastichsearch output to load one index using document_id and http output to update…
Carlitoz
- 1
- 1
0
votes
0 answers
I need to transfer dataset from power bi service to elasticsearch through Logstash. it not send any data but create Index in the Elasticsearch
i create REST API for the power bi dataset.
It is my config file to transfer data from the power bi service to elasticsearch :
input {
http_poller {
urls => {
powerbi_data => {
method => get
url =>…
Mahendra kanoujiya
- 11
- 2
0
votes
1 answer
Null value not inserted in Postgresql by Logstash
i am trying to insert data from Elasticsearch to postgresql. it insert data in the postgresql but does not handle Blank value properly. As I am trying to insert black value in postgresql it fills the variable name of the Statement.
it is output in…
Mahendra kanoujiya
- 11
- 2
0
votes
1 answer
Exception. Not retrying {:exception=>#
I want to Add My CSV File to postgreSQL.but whenever i try to connect it Show Error:
[ERROR][logstash.outputs.jdbc
][main][a4f714a30e2d2cae8e83b3c2d215c3537fe40dca0495ca92cc2f50a93ba8088a]
JDBC - Exception. Not…
Mahendra kanoujiya
- 11
- 2
0
votes
1 answer
[ERROR][logstash.outputs.jdbc ] Unknown setting 'driver_library' for jdbc
I want to Add My CSV File to postgreSQL.but whenever i try to connect it Show Error:
[ERROR][logstash.outputs.jdbc ] Unknown setting 'driver_library' for jdbc
it is My Config.conf File
input {
file {
path => "C:/Users/Desktop/Input.csv"
…
Mahendra kanoujiya
- 11
- 2
0
votes
0 answers
Problem with logstash in sync postgres to elasticsearch
I have schema: postgres -> elasticsearch -> kibana
Need to sync data from postgres to elasticsearch.
Earlier on testing server I used pgsync, but my devops don't want to reconfigure the option.
Lib need enable logical decoding by setting wal_level =…
Alex
- 3
- 4
0
votes
2 answers
Logstash : Get timestamp from mysql doesn't work, convert it to string it work?
I'm using this conf file to overwrite the @timestamp field in ElasticSearch, but I automatically get an _dateparsefailure flag:
input {
jdbc {
jdbc_driver_library => "C:/path/to/mariadb-java-client.jar"
statement => "SELECT…
Winux
- 31
- 5
0
votes
0 answers
Logstash jdbc connect to mssql
Exception: ArgumentError
Stack: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/cronline.rb:60:in `initialize'
…
user19820408
- 1
- 1