Stack Overflow
Stack Overflow

Questions tagged [logstash-jdbc]

187 questions
0
votes
2 answers

Logstash JDBC adapter: Varbinary to UTF-8? (mysql to elastic import)

I'm trying to import a mysql table into elasticsearch via logstash. One column is of the type "varbinary" which causes the following error: [2018-10-10T12:35:54,922][ERROR][logstash.outputs.elasticsearch] An unknown error occurred sending a bulk…
AndroidStorm
  • 161
  • 4
  • 16
0
votes
1 answer

SQL Server Sync with Elasticsearch through Logstash - retransfer does not happens

Background: We are doing a POC of Sync SQL Server error log data to elasticsearch(ES) to bring a dashboard in kibana. I used Logstash with jdbc input plugin to move sql server table data to (ES), it was succeeded. In the log table it was around 5000…
Ganesh Ram Ravi
  • 169
  • 1
  • 10
0
votes
0 answers

After adding a config file to the logstash with a mysql driver, the data is no longer sent to the elastic

In the logstash configuration files with PostgreSQL queries already existed. After adding the config file with mysql, after some time the data ceases to arrive in the elastic, tcpdump on the outgoing port also shows the absence of data, while…
TIgor
  • 3
  • 5
0
votes
1 answer

How to setup Logstash JDBC output to write JSON field?

I want to stream data from LogStash to a MS SQL Server working table with a schema something like this: CREATE TABLE [dbo].[WorkingTable] ( [MessageId] bigint, //Auto Incrementing primary key. [Data] nvarchar(MAX) //JSON…
chrismon
  • 75
  • 7
0
votes
0 answers

call a python script from ElasticSearch daily

I have logstash scheduled to read data from database every day at 3:00 A.M.. What I want to do is whenever the index is read I want to call a Python Function defined in my Django Project that will process index and then update another index in…
Aashish Gahlawat
  • 409
  • 1
  • 7
  • 25
0
votes
1 answer

Logstash JDBC Input Plugin - DB2

I am trying to connect to DB2 database on Mainframe from my local machine. I am using JDBC input plugin. It seems the jdbc driver license file is not getting loaded. Can someone please tell me how can I load the db2jcc_license_cisuz-1.4.2.jar…
Vipul Parikh
  • 1
0
votes
1 answer

Elasticsearch fields from one index appearing into another index

I have created a few indices into elasticsearch. I have created separate elasticsearch config file per elaticsearch index. I am using JDBC driver to fetch data from two different tables of database. When I restarted logstash after I changed the…
Atul K.
  • 352
  • 1
  • 6
  • 16
0
votes
2 answers

Logstash jdbc not sending data

I'm trying to export datas from a mysql table to elastic search with logstash and the jdbc mysql driver with every process in a docker container. My problem is (with no error) their is nothing sent to elastic search. My Dockerfile : FROM…
Kalus
  • 1
  • 2
0
votes
1 answer

ora:01882 timezone region not found - jdbc_input_logstash plugin error

Oracle DB: 11.2.0.4 OJDBC version: ojdbc6.jar JDK: openjdk 1.8 LogStash version: 6.3.2-1 I am recieving following error in logstash error log [ERROR][logstash.inputs.jdbc ] Unable to connect to database. Tried 1 times…
Ysumit
  • 1
  • 1
  • 1
0
votes
0 answers

logstash input plugin for postgresql issue - duplication (ignoring last run state)

I am using jdbc plugin to fetch data from postgresql db, it seems to be work fine for entire export and i am able to pull the data, but it is not working according to saved state, everytime all of data is queried and there are lot of duplicates. I…
hsg09
  • 1
  • 1
0
votes
1 answer

Elastic is retaining only last record from logstash

Here is the select statement , which is a transaction data , every time the data is inserted into UserProfile , the old index values gets deleted from elastic jdbc { statement => "SELECT userId,salesTeam FROM UserProfile with…
Karthik Suresh
  • 367
  • 7
  • 23
0
votes
1 answer

How to map latitude and longitude values into a geo_point converting a database using the JDBC input plugin within Logstash to be used in Kibana?

I am logging a Mariadb database into elastic search using the jdbc input plugin within logstash. The objective is to make some geographical reports using Kibana. The data get indexed, yet I can't use the longitude and latitude indexed to make the…
Assem
  • 11,574
  • 5
  • 59
  • 97
0
votes
1 answer

Import data from MySQL to elasticsearch with logstash after preprocessing

I am trying to import data from MySQL to elasticsearch using logstash, everything works fine and I have all the data imported well. However, one of the fields in MySQL called "metadata" is following a specific pattern like this…
m.alsioufi
  • 5
  • 4
0
votes
0 answers

Multiple inputs on logstash jdbc for mysql cluster

For example, I have 8 dbs, like db0, db1, db2, ... , db7. And in each db, I have nearly 1024 tables sharing with the same structure, and the tables' name are table0001, table0002, table0003,..., table1024. As far as I can know, for each db and…
chaos
  • 1,359
  • 2
  • 13
  • 25
0
votes
0 answers

Logstash - Error when try to run conf fle

I try to run a conf file to use mysql database on input and on output I want to get a json-line. But an error is happening. I'm putting the error and the conf file in this question. Sending Logstash's logs to C:/logstash-6.0.1/logs which is now…
Rodrigo Calegario
  • 1
  • 1
1 2 3
…
12 13