What is the format of logstash config file

Question

Does logstash use its own file syntax in config file? Is there any parser or validator for config file syntax?

For anyone that does not use logstash but have idea about file formats here is a sample syntax:

input {
  file {
    path => "/var/log/messages"
    type => "syslog"
  }

  file {
    path => "/var/log/apache/access.log"
    type => "apache"
  }
}

You can now check configuration when calling logstash with `--configtest` switch. I also wonder what this format is (meaning is it inspired by something, or just their own idea). — nuoritoveri, Jun 27 '14 at 15:47

score 31 · Answer 1 · edited Nov 14 '21 at 10:14

The Logstash configuration file is a custom format developed by the Logstash folks using Treetop.

The grammar itself is described in the source file grammar.treetop and compiled using Treetop into the custom grammar.rb parser.

That parser is then used by the pipeline.rb file in order to set up the pipeline from the Logstash configuration.

If you're not that much into Ruby, there's another interesting project called node-logstash which provides a Logstash implementation in Node.js. The configuration format is exactly the same as with the official Logstash, though the parser is obviously a different one written for Node.js. In this project, the Logstash configuration file grammar is described in jison and the parser is also automatically generated, but could be used by any Node.js module simply by requiring that generated parser.

This might change a bit once the [Logstash Intermediate Representation](https://github.com/elastic/logstash/issues/6018) comes to life. — Val, Apr 14 '17 at 11:40
I don't think Logstash Intermediate Representation ever came to life. — Noumenon, Jul 06 '23 at 00:57

score 9 · Answer 2 · edited Dec 29 '14 at 14:35

9

You can use the following command to verify your logstash config file is valid:

bin/logstash --configtest --config <your config file>

edited Dec 29 '14 at 14:35

guneykayim

5,210
2
29
61

answered Dec 29 '14 at 13:01

Zanbel

282
2
6

score 6 · Accepted Answer · answered Aug 31 '17 at 23:06

Apparently command line arguments have been updated since the answers have been posted and --configtest and --config arguments are no longer valid. In order to ask Logstash (at least v5) to validate config file:

bin/logstash -t -f config.conf

With expanded arguments it looks like this:

bin/logstash --config.test_and_exit --path.config config.conf

score 1 · Answer 4 · edited May 20 '15 at 14:12

1

So far. there is no any parser or validator for logstash config. You can only use the logstash to verify the config.

For more information about config, you can visit here. All the format is introduced in this page.

edited May 20 '15 at 14:12

pmod

10,450
1
37
50

answered Feb 03 '14 at 11:11

Ban-Chuan Lim

7,840
4
35
52

score 1 · Answer 5 · answered Apr 02 '19 at 21:22

The Logstash-Config Go package config provides a ready to use parser and Abstract Syntax Tree for Logstash configuration files in Go.

The basis of the grammar for the parsing of the Logstash configuration format is the original Logstash Treetop grammar .

logstash-config uses pigeon to generate the parser from the PEG (parser expression grammar).

What is the format of logstash config file

5 Answers5

Linked