How to get first char of a message in logstash filter

Question

I would like to get the firs char of the message in order to apply either xml or json filter but I don't even know how to start ```

filter {
  if [type]=="mom_rubens" {
    if [message] = "<*" {
      xml {
        source => "message"
        store_xml => false
        xpath => [
          "/APIOS_MOM_EVENT/IDENT/NO_EMIARTE/text()", "NO_EMIARTE",
          "/APISTAT_EVENT/IDENT/NO_EMIARTE/text()", "NO_EMIARTE",
          "/APIOS_MOM_EVENT/DATA/APIOS_EXPORT/METADATA/VECTORS/VECTOR","VECTORS",
          "/APIOS_MOM_EVENT/DATA/APIOS_EXPORT/METADATA/VECTORS/VECTOR/@NAME","VECTOR_NAME",
          "/APIOS_MOM_EVENT/INFO_EVENT/SENDER/text()","SENDER",
          "/APISTAT_EVENT/INFO_EVENT/SENDER/text()","SENDER",
          "/APIOS_MOM_EVENT/DATA/APIOS_EXPORT/METADATA/VECTORS/VECTOR/@ONLINE","ON_LINE",
          "/APIOS_MOM_EVENT/DATA/APIOS_EXPORT/METADATA/VECTORS/VECTOR/@OFFLINE","OFF_LINE"
        ]
        target => "xml"
      }
    }
    else if [message] = "{*" {
      json {
        source => "message"
      }
    }
  }

```

if [message] = "

```

Thanks for your help

Best regards, Guillaume

Maybe something like ```if [type]=="mom_rubens" { if ([message] =~ /^) { xml { ... } else if ([message] =~ /^{/) { json { source => "message" } } }``` but it seems that json is always ignored — glmrenard, Jan 13 '16 at 13:10
I have it :) but even if message is in kibana, it's not filtered ``` if ([message] =~ "^\<*") { xml { ``` — glmrenard, Jan 13 '16 at 13:32

score 1 · Accepted Answer · answered Jan 13 '16 at 14:55

1

Should be:

if [message] =~ /^<xml/ {
    ...
}

answered Jan 13 '16 at 14:55

Alain Collins

16,268
2
32
55

Thansk Alain but don't I need to escape < or { ? for now I don't have any parse in my json stuff – glmrenard Jan 14 '16 at 08:14

score 0 · Answer 2 · answered Jan 14 '16 at 12:56

I think it's good

filter {
  if [type]=="mom_rubens" {
    if ([message] =~ /^</) {
      xml {
        add_field => { "genre" => "xml" }
        source => "message"
        store_xml => false
        xpath => [
          "/APIOS_MOM_EVENT/IDENT/NO_EMIARTE/text()", "NO_EMIARTE",
          "/APISTAT_EVENT/IDENT/NO_EMIARTE/text()", "NO_EMIARTE",
          "/APIOS_MOM_EVENT/DATA/APIOS_EXPORT/METADATA/VECTORS/VECTOR","VECTORS",
          "/APIOS_MOM_EVENT/DATA/APIOS_EXPORT/METADATA/VECTORS/VECTOR/@NAME","VECTOR_NAME",
          "/APIOS_MOM_EVENT/INFO_EVENT/SENDER/text()","SENDER",
          "/APISTAT_EVENT/INFO_EVENT/SENDER/text()","SENDER",
          "/APIOS_MOM_EVENT/DATA/APIOS_EXPORT/METADATA/VECTORS/VECTOR/@ONLINE","ON_LINE",
          "/APIOS_MOM_EVENT/DATA/APIOS_EXPORT/METADATA/VECTORS/VECTOR/@OFFLINE","OFF_LINE"
        ]
        target => "xml"
      }
    }
    else if ([message] =~ /^{/) {
      json {
        add_field => { "genre" => "json" }
        source => "message"
      }
    }
  }

but in the json data, I have a 'type' field which cause me a lot of trouble as it overwrites my original 'type field' (mom_rubens is no more once the json is parsed)

Do I have a way to rename a field inside the json

{"sender":"opa","type":"update","programId":"065491-000-A","emNumber":"065491-000","reassembly":"A","programCaseCode":452,"genrePressCode":0,"kind":"SHOW","parents":[],"routingKey":"update.INTERNET.K_SHOW.ALW.PRG_ANG.PRG_ESP.C452.G0","platforms":["ALW","PRG_ANG","PRG_ESP"],"date":"2016-01-14T13:49:40+0100"}

In this case, I would like to have typemessage and no type

Regards,

The json{} filter has a 'target' parameter which will put the fields somewhere other than at the root level. — Alain Collins, Jan 14 '16 at 14:59
Thanks Alain, Yes I know but I cheated and now I use a search filter (genre:"json" OR genre:"xml") in order to have what I want — glmrenard, Jan 14 '16 at 15:31

How to get first char of a message in logstash filter

2 Answers2