Deserialize JSON array of arrays into List of Tuples using Newtonsoft

Question

I am receiving data that looks like this from an online service provider:

{
  name: "test data",
  data: [
    [ "2017-05-31", 2388.33 ],
    [ "2017-04-30", 2358.84 ],
    [ "2017-03-31", 2366.82 ],
    [ "2017-02-28", 2329.91 ]
  ],
}

I would like to parse it into an object that looks like this:

public class TestData
{
   public string Name;
   public List<Tuple<DateTime, double>> Data;
}

The only thing I have been able to find is how to parse an array of objects into a list of tulples, for example: Json.NET deserialization of Tuple<...> inside another type doesn't work?

Is there a way to write a custom converter that would handle this?

Why tuples? They're kinda awful in most cases (except C#7, but that's a different thing really) — DavidG, Aug 03 '17 at 15:37
A class is fine too... but it's a simple way to define something that holds two "different" datatypes... either way would be fine. — Brian Rice, Aug 03 '17 at 15:48
@BrianRice: In general a small class can be written that holds the same data and has more useful names for it. Item1 and Item2 are never going to make for readable code... There are occasionally times when you want functionality of Tuples (eg its equality logic) but more often than not a small named Class will work perfectly and be more readable. — Chris, Aug 03 '17 at 16:39
The only (slight if any) advantage is that I have had to only write one Generic converter... and I could use it for any possible types of paired data... you will also see that whenever I use tuples, I immediately pull the items off as named variables... this helps for maintenance... but, I do hear what you're saying. — Brian Rice, Aug 03 '17 at 21:39

score 5 · Answer 1 · answered Nov 28 '17 at 05:23

If anyone is interested in a more generic solution for ValueTuples

public class TupleConverter : JsonConverter
{
    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        var type = value.GetType();
        var array = new List<object>();
        FieldInfo fieldInfo;
        var i = 1;

        while ((fieldInfo = type.GetField($"Item{i++}")) != null)
            array.Add(fieldInfo.GetValue(value));

        serializer.Serialize(writer, array);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        var argTypes = objectType.GetGenericArguments();
        var array = serializer.Deserialize<JArray>(reader);
        var items = array.Select((a, index) => a.ToObject(argTypes[index])).ToArray();

        var constructor = objectType.GetConstructor(argTypes);
        return constructor.Invoke(items);
    }

    public override bool CanConvert(Type type)
    {
        return type.Name.StartsWith("ValueTuple`");
    }
}

Usage is as follows:

var settings = new JsonSerializerSettings();
settings.Converters.Add(new TupleConverter());

var list = new List<(DateTime, double)>
{
    (DateTime.Now, 7.5)
};
var json = JsonConvert.SerializeObject(list, settings);
var result = JsonConvert.DeserializeObject(json, list.GetType(), settings);

score 2 · Answer 2 · answered Aug 03 '17 at 15:49

Rather than use tuples, I would create a class that is specific to the task. In this case your JSON data comes in as a list of lists of strings which is a bit awkward to deal with. One method would be to deserialise as List<List<string>> and then convert afterwards. For example, I would go with 3 classes like this:

public class IntermediateTestData
{
    public string Name;
    public List<List<string>> Data;
}

public class TestData
{
    public string Name;
    public IEnumerable<TestDataItem> Data;
}

public class TestDataItem
{
    public DateTime Date { get; set; }
    public double Value { get; set; }
}

Now deserialise like this:

var intermediate = JsonConvert.DeserializeObject<IntermediateTestData>(json);

var testData = new TestData
{
    Name = intermediate.Name,
    Data = intermediate.Data.Select(d => new TestDataItem
    {
        Date = DateTime.Parse(d[0]),
        Value = double.Parse(d[1])
    })

};

This is the right way to go about it. However, I would use JArray and JObject instead of List and stríng. — Chris Jensen, Jul 25 '19 at 14:52

score 2 · Answer 3 · answered Aug 03 '17 at 15:59

So using JSON.NET LINQ, I managed to get it to work as you prescribed...

var result = JsonConvert.DeserializeObject<JObject>(json);
var data = new TestData
{
    Name = (string)result["name"],
    Data = result["data"]
        .Select(t => new Tuple<DateTime, double>(DateTime.Parse((string)t[0]), (double)t[1]))
        .ToList()
};

This is the full test I wrote

public class TestData
{
    public string Name;
    public List<Tuple<DateTime, double>> Data;
}

[TestMethod]
public void TestMethod1()
{
    var json =
    @"{
        name: ""test data"",
        data: [
        [ ""2017-05-31"", 2388.33 ],
        [ ""2017-04-30"", 2358.84 ],
        [ ""2017-03-31"", 2366.82 ],
        [ ""2017-02-28"", 2329.91 ]
        ],
    }";

    var result = JsonConvert.DeserializeObject<JObject>(json);
    var data = new TestData
    {
        Name = (string)result["name"],
        Data = result["data"]
            .Select(t => new Tuple<DateTime, double>(DateTime.Parse((string)t[0]), (double)t[1]))
            .ToList()
    };

    Assert.AreEqual(2388.33, data.Data[0].Item2);
}

However, while this may work, I am in agreement with the rest of the comments/answers that using tuples for this is probably not the correct way to go. Using concrete POCO's is definitely going to be a hell of a lot more maintainable in the long run simply because of the Item1 and Item2 properties of the Tuple<,>.

They are not the most descriptive...

I agree... the whole point was to turn something that is an array of arrays into a List of (some kind of) object... either a tuple or a class... your technique works either way. — Brian Rice, Aug 03 '17 at 16:38

Brian Rice · Accepted Answer · 2017-08-03T21:34:40.983

I took the generic TupleConverter from here: Json.NET deserialization of Tuple<...> inside another type doesn't work? And made a generic TupleListConverter.

Usage:

public class TestData
{
    public string Name;
    [Newtonsoft.Json.JsonConverter(typeof(TupleListConverter<DateTime, double>))]
    public List<Tuple<DateTime, double>> Data;
}

public void Test(string json)
{
    var testData = JsonConvert.DeserializeObject<TestData>(json);
    foreach (var tuple in testData.data)
    {
        var dateTime = tuple.Item1;
        var price = tuple.Item2;
        ... do something...
    }
}

Converter:

public class TupleListConverter<U, V> : Newtonsoft.Json.JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return typeof(Tuple<U, V>) == objectType;
    }

    public override object ReadJson(
        Newtonsoft.Json.JsonReader reader,
        Type objectType,
        object existingValue,
        Newtonsoft.Json.JsonSerializer serializer)
    {
        if (reader.TokenType == Newtonsoft.Json.JsonToken.Null)
            return null;

        var jArray = Newtonsoft.Json.Linq.JArray.Load(reader);
        var target = new List<Tuple<U, V>>();

        foreach (var childJArray in jArray.Children<Newtonsoft.Json.Linq.JArray>())
        {
            var tuple = new Tuple<U, V>(
                childJArray[0].ToObject<U>(),
                childJArray[1].ToObject<V>()
            );
            target.Add(tuple);
        }

        return target;
    }

    public override void WriteJson(Newtonsoft.Json.JsonWriter writer, object value, Newtonsoft.Json.JsonSerializer serializer)
    {
        serializer.Serialize(writer, value);
    }
}

Deserialize JSON array of arrays into List of Tuples using Newtonsoft

4 Answers4

Linked