Logstash File Input stream events from Files
Questions tagged [logstash-file]
221 questions
0
votes
2 answers
How to add country name field based on mobile number using Mobile_Number using logstash
I am having logs in txt file contains fields below. I want to extract country code from mobile number and create (translate) into new field called country.
I checked with translate filter but not getting how to extract country code digits from…
Divyank
- 811
- 2
- 10
- 26
0
votes
1 answer
Using bitwise comparation in logstash
How can I use if conditions to check for bitwise existence?
I have an Flag's enum (in c#) and I want to do different operations in logstash according to its value.
Phil Dunphy
- 3
- 3
0
votes
0 answers
exception logs not present in file logs
In spring boot project the logback is configured as below..Now the problem is if there is any exception its not present in logged file but its showing in stdout.
dead programmer
- 4,223
- 9
- 46
- 77
0
votes
1 answer
how to filter the values correctly, by the beginning of the sentence? Using Pattern from Logstash
I have the follow string for filter :
action=Received type=Event IdService=45221
I try this and get first :
=%{DATA:action}\s
{
"action": "Received"
}
But When i try continue
=%{DATA:action} =%{DATA:type}\s
it happens this
{}
How i must…
Hakyon
- 9
- 3
0
votes
1 answer
How to check key value pair in json string for message field in logstash
Here is sample of log json string, message field is again a json string.
{
"service_id" => "sec-sip",
"@version" => "1",
"logplane" => "containerlogs",
"componentName" => "container",
"message" =>…
user3460330
- 165
- 3
- 11
0
votes
1 answer
how to get only date from timestamp in logstash?
I have date format 12/1/2020 12:08:27 AM, I want only date like 12/1/2020. Is any way to get date in logstash? I have tried to find out the date but didn't get, What I want then I separated each term like day,month, year. But Actually I want…
Padam Rai
- 13
- 3
0
votes
0 answers
Logstash mixed log parsing
I'm processing log files through Logstash that have single-line messages and multi-line messages of various patterns, I'd like to read the multi-line messages as single-line messages and the single-line messages as is.
Currently, I'm using the below…
Sandesh
- 33
- 3
0
votes
1 answer
How to create multi indexes in .conf file in logstash
I used the following .conf file but it gives me an error.
My idea was to add more conditions in filter in order to have multi index depending on the csv file.
input {
file {
path => "/home/aitor/RETO8/*.csv"
start_position =>…
aitor
- 37
- 6
0
votes
1 answer
logstash with hdfs for paritcular duration
Hi I am new logstash and i have done with read the data from tcp and write to the hdfs...that part is don but i want to write to data to 4 different folder of hdfs
Here is sample code
input {
tcp {
host => "X.X.X.X"
port => 5051
codec =>…
sanchit
- 3
- 2
0
votes
1 answer
Load multiple increasing json files by ELK stack
I crawled a lot of JSON files in data folder, which all named by timestamp (./data/2021-04-05-12-00.json, ./data/2021-04-05-12-30.json, ./data/2021-04-05-13-00.json, ...).
Now I'm tring to use ELK stack to load those increasing JSON files.
The JSON…
jjdblast
- 525
- 1
- 8
- 26
0
votes
0 answers
Logstash Config how to trasfer aws s3 csv without header to Elasticsearch
I have sample csv file in s3 with 3 column without any header. But during data transfer from s3 csv to elasticsearch, I want to give some name to each column (in my case id, name, age to column 0 to 2 respectively).
Input Sample.csv
1,myname,23
…
Piyush N
- 742
- 6
- 12
0
votes
0 answers
Is the below logstash.conf file format correct?
Is the below Logstash.conf file format correct? If not please let me know where I am wrong. Based on the log file path(inbuilt key) I want to change the index to another
input {
beats {
port => 5044
}
}
filter {
grok {
match =>…
0
votes
1 answer
Filter per output in multiple hosts in output plugin logstash
I would like to send logs to multiple hosts in the output plugin of logstash. Is there a way I can filter logs for each output separately using the filter plugin?
Thanks in advance
Charith sattiva
- 11
- 1
0
votes
1 answer
Custom Grok Pattern for [serverity]MMDD
I'm a beginner in writing grok patterns and I'm unable to figure out how to write custom grok pattern for this
I0224 22:37:20.377508 2437 zookeeper_watcher.cpp:326] Zk Session
Disconnected, notifying watchers
"I" being log_severity. and "0224" is…
0
votes
1 answer
Multi line log not in the right order
I have a log that looks like this:
1613635264 host1 AAA 0.11 0.09 0.12 16 13
1613635264 host1 BBB 0.21 0.23 176141
1613635264 host2 AAA 2.08 1.76 1.38 4 3
1613635264 host2 BBB 6.21 0.12 228981
1613635264 host3 AAA 0.58 1.12 1.75 16 0
1613635264…
Daniel
- 341
- 6
- 24