0

Question

is there a solution for filtering out server-side logs during development (for node.js or Scala), such that you can select which log 'topics' you wish to view at a given moment, out of everything that was logged by your application?

Elaboration

I would like to be able to control which log messages I see every time I run my code in non-production mode. E.g. toggle which groups of messages I see per the task I am developing or testing, not by the standard categories like warn, info, etc. Is this scenario maybe met by graylog2's web interface? or in some other way? (if graylog - does the graylog interface allow filtering out all log messages prior to restarting my application, without going into typing in timestamps?)

Why configuring what to log isn't the answer

If there's no way/tool that facilitates that, I'll digress to managing it through my own application's configuration file, but that would filter messages at runtime, not facilitate viewing subsets of what's already been logged, thus not really useful for development. I don't mind of course assigning each of my log messages a type value for this sake. This isn't about what to log where - but about how to view logged information through a dynamic filter!

matanster
  • 15,072
  • 19
  • 88
  • 167

2 Answers2

1

Graylog2 is able to do that: You can filter either using the query language or streams. Streams allow you to match messages into categories/topics when they are received and you can be alerted in case of unusual conditions.

You can then even drag a frame in any chart of the web interface and search only in the timeframe selected.

The query language is described here: http://support.torch.sh/help/kb/graylog2-web-interface/the-search-bar-explained

I suggest you give it a spin. A RC release of v0.20.0 has been released just a few days ago and improved the installation process a lot.

Lennart Koopmann
  • 20,313
  • 4
  • 26
  • 33
  • Thanks. I expended some time on trying logstash, and it's not that brilliant in that, mainly usability quirks all-over (nor in connecting well from node.js or in having useful documentation for the various connection definitions). The query language for graylog indeed looks very nimble, if I understand correctly you can query by any elasticsearch field that came from your logging data (?). – matanster Jan 17 '14 at 20:34
  • Yes, exactly! You can either extract data by rules in Graylog2 or send already structured data using GELF: http://graylog2.org/gelf#libraries – Lennart Koopmann Jan 17 '14 at 22:33
  • To make it really useful, does gelf (and hence graylog) also allow embedding my own json object such that it is automatically mapped in the elasticsearch backend as json rather than as a string? I recall gelf is a rather limited format... – matanster Jan 18 '14 at 10:10
  • You can add any number of your own fields with all types that JSON supports. – Lennart Koopmann Jan 20 '14 at 10:50
0

Don't know about node.js but for Java/Scala one would usually use JMX for this. Here's some JMX docs from logback: http://logback.qos.ch/manual/jmxConfig.html

ed.
  • 2,696
  • 3
  • 22
  • 25
  • Anything java can be reused by/from Scala. Does logback have a UI where you can filter messages already logged through dynamic criteria, or does it only allow filtering what gets logged? Thanks – matanster Jan 17 '14 at 07:24