JSON vs Key-Value for Splunk

Question

Rolling out splunk I'm debating switching to JSON. Splunk supports spath now and even endorses JSON towards user friendliness (ref: http://dev.splunk.com/view/logging-best-practices/SP-CAAADP6 )

Ironically Splunk also recommends against JSON (ref: http://docs.splunk.com/Documentation/Storm/Storm/User/Bestpractices ). Granted storm is cloud and not hosted but wtf?

Has anyone leveraged JSON within splunk? Can someone talk towards real world performance differences in index search and regarding the ability to easily create search queries leveraging spath?

Answer 1

Splunk speaks JSON natively - so long as your JSON is not malformed, Splunk'll take it just fine

There's no appreciable difference in indexing JSON vs straight text data with Splunk

Some docs.Splunk references for your edification:

• https://docs.splunk.com/Documentation/Splunk/8.0.3/Knowledge/Automatickey-valuefieldextractionsatsearch-time
• https://docs.splunk.com/Documentation/Splunk/8.0.3/SearchReference/Spath
• https://docs.splunk.com/Documentation/Splunk/8.0.3/Data/IFXandHEC

There's nothing that says you must use spath for JSON data, btw: I often end up using multivalue operations (like mvexpand (when not in an eval)) as they're simpler and easier to understand

Answer 2

In my experience, Splunk + JSON performance is fine in later versions of Splunk. Being able to dump whatever you want out as JSON and not having to worry about escaping etc. makes the whole thing even easier. AFAIK Storm was a predecessor to Splunk, and Splunk has evolved since then.

If you're using Java & Logback, you can use a library like logstash-logback-encoder. The LoggingEventCompositeJsonEncoder encoder allows you to customise what's written out so it works well with anything, not just Logstash.