Newtonsoft JSON/JSON.NET custom serialization - System.Object members not correctly deserialized even when setting TypeNameHandling

Question

I'm trying to implement custom serialization using Newtonsoft.Json where I want all fields serialized, and then deserialized to their proper types. The class contains a field of type "object", implements ISerializable, has a [Serializable] attribute set, and has a serialization constructor. I set the value of this object field to an instance of some class, then serialize it. I use a JsonSerializer with TypeNameHandling set to TypeNameHandling.Auto.

Here's the code that I'm trying out:

using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
using System;
using System.IO;
using System.Runtime.Serialization;

namespace ConsoleApp1
{
    class Program
    {
        [Serializable]
        class Foobar : ISerializable
        {
            public Foobar()
            {
            }

            public Foobar(SerializationInfo info, StreamingContext context)
            {
                something = info.GetValue("something", typeof(object));
            }

            public void GetObjectData(SerializationInfo info, StreamingContext context)
            {
                info.AddValue("something", something);
            }

            public object Something { get { return something; } set { something = value; } }
            private object something;

            [JsonIgnore]
            private string someOtherObject = "foobar";
        }

        class SomeOtherClass
        {
            public string Foo { get; set; }
            public string Bar { get; set; }
        }

        static void Main(string[] args)
        {
            Foobar myObj = new Foobar();

            myObj.Something = new SomeOtherClass() { Bar = "My first", Foo = "fqwkifjwq" };

            var serializer = new Newtonsoft.Json.JsonSerializer();
            serializer.TypeNameHandling = TypeNameHandling.Auto;
            serializer.Formatting = Formatting.Indented;
            string serialized;
            using (var tw = new StringWriter())
            {
                using (var jw = new JsonTextWriter(tw))
                    serializer.Serialize(jw, myObj);

                tw.Flush();
                serialized = tw.ToString();
            }

            Foobar deserialized;
            using (var rt = new StringReader(serialized))
            using (var jsonReader = new JsonTextReader(rt))
                deserialized = serializer.Deserialize<Foobar>(jsonReader);

            Console.WriteLine("Type of deserialized.Something: " + deserialized.Something.GetType().FullName);
        }
    }
}

After deserialization, the field Foobar.Something is simply a Newtonsoft.Json.Linq.JObject, which is NOT what I want. I would like this to be properly deserialized to an object of type SomeOtherClass. The serialized output does contain the required info:

{
  "something": {
    "$type": "ConsoleApp1.Program+SomeOtherClass, ConsoleApp1",
    "Foo": "fqwkifjwq",
    "Bar": "My first"
  }
}

Here's what I've tried so far:

1. Using the code above. Does exactly how I describe above.
2. Using the [JsonObject(MemberSerialization = MemberSerialization.Fields)] attribute on the object I'm serializing. Then I get an object of the correct type on the Foobar.Something field (a SomeOtherClass instance), BUT, any non-serialized fields with a default value gets initialized to null instead of their default value (like the field Foobar.someOtherObject).
3. Removing the Serializable attribute. Then the ISerializable GetObjectData and the serialization constructor is not called.
4. Setting the JsonSerializer's TypeNameHandling property to All (no effect).

So - any tips on how this can be solved?

Answer 1

So, to summarize your question/comments:

1. You have a huge and complex legacy data structure which you would like to serialize and deserialize.
2. Much of the data you want to de/serialize is in private fields and it is too cumbersome to add attributes to those or add public properties.
3. Some of those fields are of type object and you want to preserve the type of those values for the round trip.
4. Some of those fields are expressly marked to be ignored for serialization purposes and they have default values which you would like to keep.
5. You tried using TypeNameHandling.Auto and implementing ISerializable; this worked for serialization (the child object type was written to the JSON) but did not work for deserialization (you got a JObject back instead of the actual child object instance).
6. You tried using TypeNameHandling.Auto and marking your outer class with MemberSerialization.Fields; this worked for serialization, but on deserialization you lost the default values for your ignored fields.

Let's look at both approaches and see what we can do.

ISerializable

It does seem a bit odd that Json.Net does not respect the TypeNameHandling.Auto setting on deserialization for child objects of ISerializable when it does write the type information for those into the JSON on serialization. I don't know whether this is a bug, an oversight, or if there some technical limitation in play here. Regardless, it does not behave as expected.

However, you may be able to implement a workaround. Since you do get a JObject back in the SerializationInfo, and that JObject has the $type string in it, you could make an extension method which will create your child object from the JObject based on the $type:

public static class SerializationInfoExtensions
{
    public static object GetObject(this SerializationInfo info, string name)
    {
        object value = info.GetValue(name, typeof(object));
        if (value is JObject)
        {
            JObject obj = (JObject)value;
            string typeName = (string)obj["$type"];
            if (typeName != null)
            {
                Type type = Type.GetType(typeName);
                if (type != null)
                {
                    value = obj.ToObject(type);
                }
            }
        }
        return value;
    }
}

Then, use it in your serialization constructor in place of GetValue whenever the type is object:

public Foobar(SerializationInfo info, StreamingContext context)
{
    something = info.GetObject("something");
}

MemberSerialization.Fields

It looks like using [JsonObject(MemberSerialization = MemberSerialization.Fields)] attribute does mostly what you want except for losing the default values on some of your ignored properties. It turns out that Json.Net intentionally does not call the normal constructor when using MemberSerialization.Fields-- instead it uses the FormatterServices.GetUninitializedObject method to create a completely empty object before populating the fields from the JSON. This obviously will prevent your default values from getting initialized.

To work around this, you can use a custom ContractResolver to replace the object creation function, e.g.:

class FieldsOnlyResolver : DefaultContractResolver
{
    protected override JsonObjectContract CreateObjectContract(Type objectType)
    {
        JsonObjectContract contract = base.CreateObjectContract(objectType);
        contract.DefaultCreator = () => Activator.CreateInstance(objectType, true);
        return contract;
    }
}

Note: the above assumes all your objects will have default (parameterless) constructors. If this is not the case, you can either add them where needed (they can be private), or change the resolver to supply a different creator function depending on the type.

To use it, just add the resolver to your JsonSerializer instance:

serializer.ContractResolver = new FieldsOnlyResolver();