Cannot install gems to use in homemade filters logstash 1.5.2

Question

I need to install gems in logstash 1.5.2

I have made a filter that need the typhoeus ruby gem. I have tried to install it with gem install typhoeus and by setting the environment variable GEM_HOME to pathToLogstash-1.5.2/vendor/bundle/jruby/1.9 (as explained here). A folder typhoeus can now be found in pathToLogstash-1.5.2/vendor/bundle/jruby/1.9/gems.

Yet, when I launch logstash with my filter named "indexFilter", for instance :

#test.conf
input {file {  path => "/tmp/test.log" } }

filter { indexFilter { } }

output {stdout {}}

launching

bin/logstash -f test.conf

will results in

The error reported is:
  Couldn't find any filter plugin named 'indexFilter'. Are you sure this is correct? Trying to load the indexFilter filter plugin resulted in this error: no such file to load -- typhoeus

I have found ways to install gems with previous versions of logstash (<=1.4) here

Can anyone give me the way to do the same in logstash 1.5 ?

Thank you

Answer 1

You don't need to install the typhoeus gem manually. The new logstash plugin environment takes care of your dependencies. You just have to mention them in your code (twice).

Dependency prep

For a new filter plugin you should have a directory structure like this:

├── Gemfile
├── LICENSE
├── README.md
├── Rakefile
├── lib
│   └── logstash
│       └── filters
│           └── mypluginname.rb
├── logstash-filter-mypluginname.gemspec
└── spec
    └── filters
        └── mypluginname_spec.rb

At the top of the lib/logstash/filters/mypluginname.rb:

require "typhoeus"

And at the end of your logstash-filter-mypluginname.gemspec:

s.add_runtime_dependency "typhoeus", '>= 0.7.3'

Build & Install

When you have all those things done you can auto-load your dependencies and build the gem (make sure that your directory is initialized as a git repository):

bundle install
rake vendor
bundle exec rspec
gem build logstash-filter-mypluginname.gemspec

Afterwards install your plugin using logstash's built-in plugin manager:

LS_HOME/bin/plugin install logstash-filter-mypluginname-0.0.1.gem

Verify the installation:

LS_HOME/bin/plugin list

Final remarks

I would really recommend you to stick to logstash's How to write a filter plugin guide. I've done it a few times and in my opinion this is the best way. I've experienced some issues with bundle and jruby versions so I would recommend you to use RVM to get things done.