0

I just got ELK-stack (Elastic, Logstash and Kibana) up and running. And now I'm ready to start processing logs.

My question is: What are common ways of collecting logs for handling with logstash ?

Scenario: I have some logs on a server, that I need to get over to the logstash-server.

Thanks in advance

Answer (as I understand it)

It is up to me, to decide how the logs are collected, for handling by logstash. If you also install logstash on the servers from which the logs are to be collected, logstash has many input (to process the logs locally) and output plugins (to ship the logs) that will make it possible to ship logs to the central logstash server(s).

2 Answers2

0

Logstash is a flexible too for reading and parsing logs. We use logstash on our machines to read the local logs and send it to elasticsearch. Later, we view the data in Kibana.

So, to answer you question. You can use the input plugins, which can be found here: https://www.elastic.co/guide/en/logstash/current/input-plugins.html

To connect the logstash to elasticsearch, you can find the output plugins at: https://www.elastic.co/guide/en/logstash/current/output-plugins.html

Depending on what you want to use on the server which contains the logs. You could always put another logstash instance on it, or add a logstash forwarder, or use something like apache nifi (http://nifi.incubator.apache.org/) to stream the logs back to your logstash server.

Hope this helps

deanw
  • 95
  • 1
  • 9
  • Sorry, not quite. I understand what the structure of the elk-stack is. But it is not clear to me how the logs are collected. –  Jul 08 '15 at 17:27
  • Updated my answer to hopefully be a little more clear. – deanw Jul 08 '15 at 17:57
0

I would ask myself what kind of existing log capabilities do I have on my servers, for example you might already have tools that is capable of sending logs from the server somehow, might be that syslog is running, might be that you have SNMP traps, tools from comapnies like HP or Microsoft, running docker or whatever.

The important part is that logstash will most likley be able to take your logs, its more about what your severs are able to send and without changing so mutch on the server side.

For example on windows systems NXLog is really great, but that means that you have to install it on all servers.

Anders Johansson
  • 3
  • 1