4

I am running logstash 1.5.0.1 and I have multiple config files in my /etc/logstash/conf.d folder:

 01-input-source-one.conf
 02-input-source-two.conf
 10-filter-one.conf
 11-filter-two.conf
 20-output-one.conf
 21-output-two.conf

After modifying a config file I test using /opt/logstash/bin/logstash agent -f /etc/logstash/conf.d --configtest.

If there is a problem with one of my config files I get a message that references the line number in the combined file as follows:

Error: Expected one of #, ", ', } at line 331, column 13 (byte 15167) after filter {

Sometimes, but not all the time, it shows a code snippet after the error that helps me locate the correct file but the line number in the error still doesn't do me any good.

Does anyone know if there is a way to view the combined config file?

This could also be useful in troubleshooting configuration issues that aren't the result of a syntax error (that pass a --configtest) but the result of an ordering issue.

Thanks!

Peter M
  • 973
  • 2
  • 15
  • 27

3 Answers3

2

Yes, run it interactively from the command line. It will generate a lot of other information though.

/opt/logstash/bin/logstash --debug -f /etc/logstash/conf.d

Or run following to just test and see the compiled config

/opt/logstash/bin/logstash --debug --debug-config -t -f /etc/logstash/conf.d/

You probably want to tee the output, this also gives you all of the grok filters etc.

riemann
  • 53
  • 3
TheFiddlerWins
  • 2,999
  • 1
  • 15
  • 22
  • Thanks for the quick reply! That helps me with the second part of my question because I can see the whole config, but it doesn't help me track down the syntax errors when they are referenced by a line number in the --configtest output.... – Peter M Sep 03 '15 at 19:09
  • 2
    Run a test against each file one at a time (so -f /etc/logstash/conf.d/filename – TheFiddlerWins Sep 03 '15 at 20:41
0

cat -n /etc/logstash/conf.d/*

This will print the contents of all the files in that folder in order with line numbers as one continuous file so the line number in the logstash debug will correlate.

Mat
  • 1
0

Beware, at least in version 6.3.0, I found that if you specify -f or --path.config as a directory, it will include all files under that directory (even including things like, README files).

Ensure you specify .../conf.d/*.conf in your command-line arguments (if used), and also in pipelines.yml.

In my deployment, this is how I can debug my config (version 6.3.0 -- note the arguments are a bit different to what has been provided above)

/usr/share/logstash/bin/logstash \
  --path.settings /etc/logstash/enrichment/ \
  --debug --config.debug -t

(Note that I need to set by --path.settings because I have multiple logstash instances running)

So you could get the config as logstash would get it and then add line numbers....

/usr/share/logstash/bin/logstash \
  --path.settings /etc/logstash/enrichment/ \
  --debug --config.debug -t 2>&1 \
  | grep -v '^\[' \
  | cat -n

But that's pretty slow, but useful to know if something faster doesn't work, such as:

cat /etc/logstash/enrichment/conf.d/*.conf | cat -n

Just remember to ask for the same set of files that logstash is asking for; specifying .../conf.d is not the same as specifying .../conf.d/*.conf

Cameron Kerr
  • 4,069
  • 19
  • 25