Deserializing embedded array/list to array in c# as expandoobject

Question

I have JSON objects having embedded arrays - with no predefined strongly typed class to deserialize to. ExpandoObject deserialization with Json.Net works, but the array is deserialized to list, which is an issue for me. I need expndoobject with arrays. Is there any setting I could use with Json.NET to achieve this?

Example:

var obj = """
{
    "name": "John",
    "age": 18,
    "grid": [
        {
            "type": "A",
            "price": 13
        },
        {
            "type": "B",
            "price": 1
        },
        {
            "type": "A",
            "price": 17
        }
    ]
}
""";

var engine = new Engine()
   .Execute("function eval(value) { return value.grid.filter((it)=>it.type === 'A').map(it=>it.price).reduce((a,b)=>a+b) }");

dynamic v = JsonConvert.DeserializeObject<ExpandoObject>(obj, new ExpandoObjectConverter());

engine.Invoke("eval", v);

Where this library is used: https://github.com/sebastienros/jint Result:

And I need an array there, or otherwise the call fails ("Property 'filter' of object is not a function").

Using dynamic v= Newtonsoft.Json.Linq.JObject.Parse(obj); I got this:

And still fails with: "Accessed JArray values with invalid key value: "filter". Int32 array index expected."

If I define classes for this sample:

class Inner
{ 
    public string Type { get; set; }
    public int Price { get; set; }
}

class X
{
    public string Name { get; set; }
    public int Age { get; set; }
    public Inner[] Grid { get; set; }
}

it is parsed just fine (var v = JsonConvert.DeserializeObject<X>(obj);) and the code returns what I am expecting. Not so when I use List<Inner> instead of the array. Hence the problem is that it is not an array. So I am looking for any solution that results in an array at that position.

Answer 1

why don't try something like this

Inner[] inners = JObject.Parse(obj).Properties()
            .Where( p=> p.Value.Type== JTokenType.Array)
            .SelectMany(p => p.Value.ToObject<Inner[]>())
            .ToArray();

Answer 2

This is a Jint issue. The latest stable version won't even parse the JS function. The following code, using the latest stable 2.11.58, throws without any data:

using Jint;

var js = """
    function eval(value) { 
        return value.grid.filter((it) => it.type === 'A')
                         .map(it => it.price)
                         .reduce((a,b)=>a+b) 
    }
    """;

var engine = new Engine().Execute(js);

This throws

Jint.Parser.ParserException
  HResult=0x80131500
  Message=Line 2: Unexpected token >
  Source=Jint
  StackTrace:
   at Jint.Parser.JavaScriptParser.ThrowError(Token token, String messageFormat, Object[] arguments)
   at Jint.Parser.JavaScriptParser.ThrowUnexpected(Token token)
   at Jint.Parser.JavaScriptParser.ParsePrimaryExpression()
   at Jint.Parser.JavaScriptParser.ParseLeftHandSideExpressionAllowCall()
...

The latest stable doesn't understand arrow functions to begin with.

The latest 3.0 beta, 3.0.0-beta-2044 can parse this but throws with a different error than the one in the question. I guess the latest beta has progressed a bit. This time filter is recognized but it doesn't work yet.

Invoking the function with data

var obj = """
{
    "name": "John",
    "age": 18,
    "grid": [
        {
            "type": "A",
            "price": 13
        },
        {
            "type": "B",
            "price": 1
        },
        {
            "type": "A",
            "price": 17
        }
    ]
}
""";

dynamic v = JsonConvert.DeserializeObject<dynamic>(obj);

engine.Invoke("eval", v);

throws

System.ArgumentException
  HResult=0x80070057
  Message=Accessed JArray values with invalid key value: "filter". Int32 array index expected.
  Source=Newtonsoft.Json
  StackTrace:
   at Newtonsoft.Json.Linq.JArray.get_Item(Object key)
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
--- End of stack trace from previous location ---
   at Jint.Runtime.ExceptionHelper.ThrowMeaningfulException(Engine engine, TargetInvocationException exception)
   at Jint.Runtime.Interop.Reflection.ReflectionAccessor.GetValue(Engine engine, Object target)
   at Jint.Runtime.Descriptors.Specialized.ReflectionDescriptor.get_CustomValue()
...

In this case, Jint tried to use filter as an index value for grid.

Array access does work though, so it's not JArray that's causing the problem.

This JS function :

var js = """
    function eval(value) { 
        return value.grid[0].type 
    }
    """;

Works and returns A

Answer 3

I have created a modified version of the original ExpandoObjectConverter. And that works.

public static class _
{
    public static bool MoveToContent(this JsonReader reader)
    {
        JsonToken tokenType = reader.TokenType;
        while (tokenType == JsonToken.None || tokenType == JsonToken.Comment)
        {
            if (!reader.Read())
            {
                return false;
            }
            tokenType = reader.TokenType;
        }
        return true;
    }

    public static bool IsPrimitiveToken(this JsonToken token)
    {
        if ((uint)(token - 7) <= 5u || (uint)(token - 16) <= 1u)
        {
            return true;
        }
        return false;
    }
}


public class MyExpandoObjectConverter : JsonConverter
    {
        /// <summary>
        /// Writes the JSON representation of the object.
        /// </summary>
        /// <param name="writer">The <see cref="JsonWriter"/> to write to.</param>
        /// <param name="value">The value.</param>
        /// <param name="serializer">The calling serializer.</param>
        public override void WriteJson(JsonWriter writer, object? value, JsonSerializer serializer)
        {
            // can write is set to false
        }

        /// <summary>
        /// Reads the JSON representation of the object.
        /// </summary>
        /// <param name="reader">The <see cref="JsonReader"/> to read from.</param>
        /// <param name="objectType">Type of the object.</param>
        /// <param name="existingValue">The existing value of object being read.</param>
        /// <param name="serializer">The calling serializer.</param>
        /// <returns>The object value.</returns>
        public override object? ReadJson(JsonReader reader, Type objectType, object? existingValue, JsonSerializer serializer)
        {
            return ReadValue(reader);
        }

        private object? ReadValue(JsonReader reader)
        {
            if (!reader.MoveToContent())
            {
                throw new Exception("Unexpected end when reading ExpandoObject.");
            }

            switch (reader.TokenType)
            {
                case JsonToken.StartObject:
                    return ReadObject(reader);
                case JsonToken.StartArray:
                    return ReadList(reader);
                default:
                    if (reader.TokenType.IsPrimitiveToken())
                    {
                        return reader.Value;
                    }

                    throw new Exception($"Unexpected token when converting ExpandoObject: {reader.TokenType}");
            }
        }

        private object ReadList(JsonReader reader)
        {
            IList<object?> list = new List<object?>();

            while (reader.Read())
            {
                switch (reader.TokenType)
                {
                    case JsonToken.Comment:
                        break;
                    default:
                        object? v = ReadValue(reader);

                        list.Add(v);
                        break;
                    case JsonToken.EndArray:
                        return list.ToArray();
                }
            }

            throw new Exception("Unexpected end when reading ExpandoObject.");
        }

        private object ReadObject(JsonReader reader)
        {
            IDictionary<string, object?> expandoObject = new ExpandoObject();

            while (reader.Read())
            {
                switch (reader.TokenType)
                {
                    case JsonToken.PropertyName:
                        string propertyName = reader.Value!.ToString()!;

                        if (!reader.Read())
                        {
                            throw new Exception("Unexpected end when reading ExpandoObject.");
                        }

                        object? v = ReadValue(reader);

                        expandoObject[propertyName] = v;
                        break;
                    case JsonToken.Comment:
                        break;
                    case JsonToken.EndObject:
                        return expandoObject;
                }
            }

            throw new Exception("Unexpected end when reading ExpandoObject.");
        }

        /// <summary>
        /// Determines whether this instance can convert the specified object type.
        /// </summary>
        /// <param name="objectType">Type of the object.</param>
        /// <returns>
        ///     <c>true</c> if this instance can convert the specified object type; otherwise, <c>false</c>.
        /// </returns>
        public override bool CanConvert(Type objectType)
        {
            return (objectType == typeof(ExpandoObject));
        }

        /// <summary>
        /// Gets a value indicating whether this <see cref="JsonConverter"/> can write JSON.
        /// </summary>
        /// <value>
        ///     <c>true</c> if this <see cref="JsonConverter"/> can write JSON; otherwise, <c>false</c>.
        /// </value>
        public override bool CanWrite => false;
    }

....
dynamic v = JsonConvert.DeserializeObject<ExpandoObject>(obj, new MyExpandoObjectConverter());

Yes, I am aware, that the JS engine part might change, as this is beta. However, my JS function is perfectly valid. I don't really expect the engine to be worse in compatibility with the standards and not better. But my error was not related to that. I simply asked for a solution to deserialize to an array instead of a list.