0

I am pretty new to logstash.

In our application we are creating multiple indexes, from the below thread i could understand how to resolve that How to create multiple indexes in logstash.conf file?

but that results in many duplicate lines in the conf file (for host, ssl, etc.). So i wanted to check if there is any better way of doing it? 

output {  
  stdout {codec => rubydebug}
  if [type] == "trial" {
    elasticsearch {  
        hosts => "localhost:9200"  
        index => "trial_indexer"   
    }
  } else {
    elasticsearch {  
        hosts => "localhost:9200"  
        index => "movie_indexer"   
    }
}

Instead of above config, can i have something like below? 

output {  
  stdout {codec => rubydebug}
  elasticsearch {  
        hosts => "localhost:9200"
  }
  if [type] == "trial" {
    elasticsearch {  
        index => "trial_indexer"   
    }
  } else {
    elasticsearch {  
        index => "movie_indexer"   
    }
}
Ankush Ganatra
  • 500
  • 5
  • 23

1 Answers1

1

What you are looking for is using Environment Variables in logstash pipeline. You define this once, and can use same redundant values like you said for HOST, SSL etc.

For more information Logstash Use Environmental Variables

e.g., 

output {
  elasticsearch{
    hosts => ${ES_HOST}
    index => "%{type}-indexer"
  }
}

Let me know, if that helps.

mohdasha
  • 311
  • 1
  • 7
  • Hi Ashif, this is not what i was looking for. what i wanted to see is, if its possible to move the common lines out of the logical conditioning – Ankush Ganatra Mar 15 '18 at 01:07
  • Didn't get you... Are you saying the one i specified? – Ankush Ganatra Mar 15 '18 at 05:14
  • The one in my answer, it's a general way to name indexes, if you have some field which separates you log types. Just try with index => "%{[type]}-indexer" or index => "%{type}-indexer". One of them will definitely work. – mohdasha Mar 15 '18 at 08:16
  • Then i think, what you are looking for is using Environment Variables in logstash pipeline. You define this once, and can use same redundant values like you said for HOST, SSL etc. For more information https://www.elastic.co/guide/en/logstash/current/environment-variables.html – mohdasha Mar 15 '18 at 17:00
  • Thanks Ashif, this might work. Can you edit your answer to this link. I will select it as correct one. – Ankush Ganatra Mar 16 '18 at 11:57
  • @AnkushGanatra Done. Thanks !! – mohdasha Mar 16 '18 at 12:18