Using filters with JsonPath in C#

Question

I'm using JsonPath for C# to query some JSON data. JsonPath doesn't come with its own parser, so as per Rick Sladkey's advice, I'm using Json.NET to parse my Json string into a collection of nested IDictionary objects, IList arrays, and primitives. Then I use JsonPath to filter it (after adding the class suggested in Rick Sladkey's answer).

For reference, here's the part of my code that actually handles all this:

// string exampleJsonString defined below
string query = "$.store.book[*].title" // we should get a list of all titles

// step 1: use Json.NET to parse the json string into a JObject
JObject parsedJson = JObject.Parse(exampleJsonString); 

// step 2: use JsonPath with a custom ValueSystem (JsonNetValueSystem)
JsonPathContext context = new JsonPathContext    
        { ValueSystem = new JsonNetValueSystem() }; 

// step 3: get results using JsonPath
List<object> toRet = context.SelectNodes(parsedJson,
        query).Select(node => node.Value).ToList();

The reason I was using JsonPath in the first place was because of its filter functionality. You can not only do normal queries like "$.store.book[*].title" (to get an array of all the titles in the book store), but also queries like "$.store.book[?(@.category == 'fiction')].title" (to get an array of all titles in the book store whose category matches 'fiction'). I need to be able to pass entire queries as a string, and so being able to filter while querying is extremely helpful.

Unfortunately, I'm having some trouble getting this filter functionality to work. I expect that I'll have to make adjustments to either the JsonNetValueSystem class (defined originally in the aforementioned stack overflow answer) or the JsonPath namespace (you can get JsonPath.cs from JsonPath's google code page). If there's some external tool or an alternative parsing mechanism to Json.NET that would allow me to keep JsonPath's filtering without having to write too much extra code, that would be ideal, but I'm pretty sure I'll have to alter either JsonNetValueSystem or JsonPath itself. (Both of these are fairly easy to alter since they're just .cs files, but I can't really dig into Json.NET without a lot more work.)

I can't actually seem to figure out where the original JsonPath code handles filtering, nor can I figure out why the JsonNetValueSystem class robs it of that functionality. Any advice for how to add the ability to filter in the query string would be very much appreciated. Even if it's just "don't mess with JsonPath, just change JsonNetValueSystem" or vice versa.

string exampleJsonString = "{
    "store": {
        "book": [ {
            "category": "reference",
            "author": "Nigel Rees",
            "title": "Sayings of the Century",
            "price": 8.95
        }, {
            "category": "fiction",
            "author": "Evelyn Waugh",
            "title": "Sword of Honour",
            "price": 12.99
        }, {
            "category": "fiction",
            "author": "Herman Melville",
            "title": "Moby Dick",
            "isbn": "0-553-21311-3",
            "price": 8.99
        }, {
            "category": "fiction",
            "author": "J. R. R. Tolkien",
            "title": "The Lord of the Rings",
            "isbn": "0-395-19395-8",
            "price": 22.99
        } ],
        "bicycle": [ {
            "color": "red",
            "price": 19.95,
            "style": [ "city", "hybrid" ]
        }, {
            "color": "blue",
            "price": 59.91,
            "style": [ "downhill", "freeride" ]
        } ]
    }
}"

Answer 1

When you use the script expression in a query (the ?(...) part), you need to provide a ScriptEvaluator method to evaluate the script. Unfortunately they don't provide a default implementation for the C# version. You'll need to provide that implementation.

Out of the box, this won't be the most trivial problem to solve, you'll need to write an interpreter. You have a couple of options: use CodeDOM to compile and execute the script as C# code (or any language you would prefer), use Roslyn to parse the script and evaluate, whatever works.

A quick and dirty solution for this particular case would be to do something like this in your script evaluator method:

object EvaluateScript(string script, object value, string context)
{
    if (script == "@.category == 'fiction'")
    {
        var obj = value as JObject;
        return (string)obj["category"] == "fiction";
    }
    return null;
}

Here's another solution which utilizes IronPython to evaluate the script.

public class IronPythonScriptEvaluator
{
    private Lazy<ScriptEngine> engine = new Lazy<ScriptEngine>(() => Python.CreateEngine());
    private ScriptEngine Engine { get { return engine.Value; } }

    private const string ItemName = "_IronPythonScriptEvaluator_item";

    public object EvaluateScript(string script, object value, string context)
    {
        var cleanScript = CleanupScript(script);
        var scope = Engine.CreateScope();
        scope.SetVariable(ItemName, value);
        return Engine.Execute<bool>(cleanScript, scope);
    }

    private string CleanupScript(string script)
    {
        return Regex.Replace(script, @"@", ItemName);
    }
}

Answer 2

Another solution is to use Manatee.Json instead. It has a native JSONPath implementation and parser all built in (along with schema and a serializer). Moreover, you aren't required to represent the path as a string. Manatee.Json has a fluent interface that you can use to build paths, including expression support.

To represent $.store.book[*].title, you would have

var path = JsonPathWith.Name("store")
                       .Name("book")
                       .Array()
                       .Name("title");

For your example $.store.book[?(@.category == 'fiction')].title, you'd use

var path = JsonPathWith.Name("store")
                       .Name("book")
                       .Array(jv => jv.Name("category") == "fiction")
                       .Name("title");

What's more is that there is limited support for fields within those expressions as well.

If your path source is a string, Manatee.Json handles path parsing, too!