Logstash configuration: http output with Base 64 encoded headers

Question

I use Logstash with File Input & Http Output to a homegrown service which requires credentials (username:password) to be sent as Base64 encoded value. Below is my Logstash configuration. Currently I can send the Base 64 encoded value via headers however I am looking for a way to encode a String (which would be available to Logstash via environment variable) to a Base 64 encoded String value. Would highly appreciate if anyone can throw some light on this.

input {
 file {
  path => "/home/tom/testData/test2.log"
  type => "log"
 }
}

filter {
 if [type] == "log" {
  grok {
    match => ["message", "%{TIMESTAMP_ISO8601:time} %{LOG:level} %{GREEDYDATA:lmessage}"]
  }
  ruby {
    code => "require 'base64';
    event['pass'] = Base64.encode64('User:Pwd')"
  }

}
}
output {
stdout { codec => rubydebug }
http {
http_method => "post"
  url => "http://10.1.2.3:1123/HomeService?User=Tom"
  format => "message"
  headers => {"Authorization" => "Basic %{pass}"}      
  content_type => "application/json"
  message => '{"Outer": { "Inner": [ {"doc": { "Timestamp": "%{time}", "Single": "%{level}","Double": "%{lmessage}" } } ] } }'
 }
}

Alcanzar · Accepted Answer · 2015-04-09T02:14:51.423

4

You can pull in something from the environment and base64 encode it like this:

    ruby {
            code => "
                    require 'base64';
                    event['password'] = Base64.encode64(ENV['USER_PASS']).sub(/\n/,'')
            "
    }

Then to use it:

 export USER_PASS="dog:cat"
 echo '{"test":1}' |  bin/logstash agent -f logstash.conf

With a config file like this:

input {
 stdin { codec => json }
}

filter {
    ruby {
        code => "
            require 'base64';
            event['password'] = Base64.encode64(ENV['USER_PASS'])
        "
    }
}
output {
    stdout { codec => rubydebug }
    http {
        http_method => "post"
        url => "http://localhost:1123"
        format => "message"
        headers => {"Authorization" => "Basic %{password}"}
        content_type => "application/json"
        message => '{"whatever": 1 }'
    }
}

You can see that it does the right thing by running nc -l 1123 and see:

POST  HTTP/1.1
host: localhost
connection: keep-alive
authorization: Basic ZG9nOmNhdA==
content-type: application/json
content-length: 15

{"whatever": 1}

Which is the right value:

echo ZG9nOmNhdA== | base64 -D
dog:cat

edited Apr 09 '15 at 02:14

answered Apr 02 '15 at 13:29

Alcanzar

16,985
6
42
59

Can I use the Ruby filter in conjunction with the grok filter I have at present? I could pull-in an environment variable even from from output filter. – John C Apr 02 '15 at 21:27
Yep. In logstash unless you stop it, events walk down the filter chain. – Alcanzar Apr 02 '15 at 21:29
What am not clear is where would the conversion from String --> Base64 encoded String occur. – John C Apr 02 '15 at 21:53
Am going to accept this, Many Thanks! Watch out for more Logtsash related questions for the benefit of logtsash growing community. – John C Apr 04 '15 at 21:34
Any idea why headers => ["Authorization","Basic %{password}"] won't work – John C Apr 09 '15 at 00:53
Did not work. Have updated the logstash configuration above. If I hardcode the %{pass} it works which tells me that the problem is reading the password from the ruby filter. Would highly appreciate any recommendations around the same. – John C Apr 09 '15 at 01:42
1

then you should be putting `Basic %{enccred}` -- I did a test and if you have event['password'] = 'something' and put the `Basic %{password}` in the config file it does work. – Alcanzar Apr 09 '15 at 01:50
1

Works now, the problem is ruby's encode64 adds a new line at the end so I had to substitute the newline with a '' – John C Apr 09 '15 at 02:09
Have added another Logstash problem, would be great if you take a look for the greater good of the community! – John C Apr 11 '15 at 01:58
Hello, i hope someone can answer me here, it seems when i try to check the packets, the value on the Authorization is Basic %{password}, it seems that it was not parsed. am i missing something, another plugin maybe? – lemoncodes Mar 26 '21 at 03:52

Logstash configuration: http output with Base 64 encoded headers

1 Answers1

Linked