22

I used the following piece of code to create an index in logstash.conf

output {  
    stdout {codec => rubydebug}  
    elasticsearch {  
        host => "localhost"  
        protocol => "http"  
        index => "trial_indexer"   
    }
}

To create another index i generally replace the index name with another in the above code. Is there any way of creating many indexes in the same file? I'm new to ELK.

kavya
  • 759
  • 4
  • 14
  • 31

1 Answers1

61

You can use a pattern in your index name based on the value of one of your fields. Here we use the value of the type field in order to name the index:

output {  
    stdout {codec => rubydebug}  
    elasticsearch {  
        host => "localhost"  
        protocol => "http"  
        index => "%{type}_indexer"   
    }
}

You can also use several elasticsearch outputs either to the same ES host or to different ES hosts:

output {  
    stdout {codec => rubydebug}  
    elasticsearch {  
        host => "localhost"  
        protocol => "http"  
        index => "trial_indexer"   
    }
    elasticsearch {  
        host => "localhost"  
        protocol => "http"  
        index => "movie_indexer"   
    }
}

Or maybe you want to route your documents to different indices based on some variable:

output {  
    stdout {codec => rubydebug}
    if [type] == "trial" {
        elasticsearch {  
            host => "localhost"  
            protocol => "http"  
            index => "trial_indexer"   
        }
    } else {
        elasticsearch {  
            host => "localhost"  
            protocol => "http"  
            index => "movie_indexer"   
        }
    }
}

UPDATE

The syntax has changed a little bit in Logstash 2 and 5:

output {  
    stdout {codec => rubydebug}
    if [type] == "trial" {
        elasticsearch {  
            hosts => "localhost:9200"  
            index => "trial_indexer"   
        }
    } else {
        elasticsearch {  
            hosts => "localhost:9200"  
            index => "movie_indexer"   
        }
    }
}
Val
  • 207,596
  • 13
  • 358
  • 360
  • are you referring to the "type" declared in "input" section? if so , is there a way to use as index one of the fields in actual redis event I'm logging? – Kepedizer Jan 08 '17 at 10:36
  • 1
    @Kepedizer `type` can be defined anywhere, whether in the input section or the filter section. The only issue [in your other question](http://stackoverflow.com/questions/41531883/logstash-wont-pass-index-from-redis) is a typo, you need to use `%{index}` instead of `${index}` – Val Jan 09 '17 at 04:39
  • @Val I was having a terrible time trying to track down the answer to this in the Logstash docs, so thank you very much for clearing it up. Score another one for StackOverflow. – Patrick Lee Mar 19 '17 at 01:31
  • `if [type] == "trial" && [msg] == "yes"` can i use and condition in output?? @Val – AATHITH RAJENDRAN Dec 24 '19 at 12:47
  • 1
    @AATHITHRAJENDRAN yes definitely – Val Dec 24 '19 at 12:55