3

I 'm newbie in ELK and and I'm getting issues while running logstash. I ran logstash as define in structure step by step as I do for file beat but

But when run filebeat and logstash, Its show logstash successfully runs at port 9600. In filebeat it gives like this

INFO No non-zero metrics in the last 30s

Logstash is not getting input from file beat. Please help.

My problem is as the same as this article and did what it said but noting change .

the filebeat.yml is :

    filebeat.prospectors:


    - input_type: log 

      paths:
        - /usr/share/tomcat/log_app/news/*.log


    output.logstash:

      hosts: ["10.0.20.163:5000"]

and I ran this command sudo ./filebeat -e -c filebeat.yml -d "publish"

the logstash config file is : 

input {
    beats {
        port => "5000"
    }
}
 filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }
    geoip {
        source => "clientip"
    }
}
output {
    elasticsearch {
        hosts => [ "localhost:9200" ]
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
        document_type => "%{[@metadata][type]}"

    }
}

then ran the commands

1)bin/logstash -f first-pipeline.conf --config.test_and_exit - this gave Ok 2)bin/logstash -f first-pipeline.conf --config.reload.automatic -This started the logstash on port 9600

I couldn't proceeds after this since filebeat gives the INFO

INFO No non-zero metrics in the last 30s

and I use elastic search : 5.5.1 kibana : 5.5.1 logstash : 5.5.1 file beat : 5.5.1

Hadii Varposhti
  • 416
  • 4
  • 22

2 Answers2

5

If you want to resend your data, you can try to delete filebeat's registry file, and when you restart filebeat, it will send the data again.

File location depends on your platform. See https://www.elastic.co/guide/en/beats/filebeat/5.3/migration-registry-file.html

Registry file location can also be defined in your filebeat.yml:

filebeat.registry_file: registry

https://www.elastic.co/guide/en/beats/filebeat/current/configuration-global-options.html

Imma
  • 481
  • 4
  • 8
0

Everytime you stop the filebeat. It will start reading the data from the tail of file. And because the sample file which you are using are not getting frequent data. It's not able to fetch and send it to elastic search.

Edit your log file. Add few more redundant data and then try it. It should work.

This error which you have mentioned is because FIlebeat is not able to get any updated data in that file.

Ashu Rawat
  • 21
  • 1