7

I'm new to ELK and I'm getting issues while running logstash. I ran the logatash as defined in below link https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html

But when run filebeat and logstash, Its show logstash successfully runs at port 9600. In filebeat it gives like this

INFO No non-zero metrics in the last 30s

Logstash is not getting input from filebeat.Please help..

the filebeat .yml is 

 filebeat.prospectors:
 - input_type: log
 paths:
 - /path/to/file/logstash-tutorial.log 
 output.logstash:
 hosts: ["localhost:5043"]

and I ran this command sudo ./filebeat -e -c filebeat.yml -d "publish"

The config file is 

input {
    beats {
         port => "5043"
     }
  }

output {
     stdout { codec => rubydebug }
}

then ran the commands

 1)bin/logstash -f first-pipeline.conf --config.test_and_exit - this        gave warnings
2)bin/logstash -f first-pipeline.conf --config.reload.automatic -This started the logstash on port 9600

I couldn't proceeds after this since filebeat gives the INFO

INFO No non-zero metrics in the last 30s

And the ELK version used is 5.1.2

sandra
  • 81
  • 1
  • 1
  • 3
  • What's your input? How many new events are being generated by that input? What's your logstash config? Have you tested basic connectivity from logstash to elasticsearch? Have you looked at log files, or turned up debug levels? – Alain Collins Jan 31 '17 at 05:23
  • I'm working as defined on the above mentioned link. – sandra Jan 31 '17 at 05:35
  • @Alain Collins -- I have edited my question.can u please help? – sandra Jan 31 '17 at 05:59
  • The details help. Is new stuff being added to the log file? – Alain Collins Jan 31 '17 at 06:27
  • @Alain Collins , No new stuff. The log file also downloaded from elastic.co that is from the above link. I'm following the same step mentioned in above link – sandra Jan 31 '17 at 06:35
  • Try deleting the `data` folder that is created in the current directory. Then run Filebeat again, but with all debug enabled using `sudo ./filebeat -e -c filebeat.yml -d "*"`. – A J Jan 31 '17 at 14:51
  • @ A J - still not working – sandra Feb 01 '17 at 11:48
  • have you tried deleting /var/lib/filebeat/registry and restarting filebeat? if it's not working take a look into debug log of filebeat and logstash – Akoya Apr 12 '17 at 12:42
  • @sandra, you should try out suggestions from comments and answers and you should reply back if it is working or not. – Hiren patel May 12 '17 at 07:28

2 Answers2

5

The registry file stores the state and location information that Filebeat uses to track where it was last reading

So you can try updating or deleting registry file. see here

cd /var/lib/filebeat
sudo mv registry registry.bak
sudo service filebeat restart

I have also faced this issue and I have solved with above commands.

Hiren patel
  • 971
  • 8
  • 25
-1

Filebeat reads from the end of your file, and is expecting new stuff to be added over time (like a log file).

To make it read from the beginning of the file, set the 'tail_files' option.

Also note the instructions there about re-processing a file, as that can come into play during testing.

Alain Collins
  • 16,268
  • 2
  • 32
  • 55