Can System.Text.Json.JsonSerializer serialize collections on a read-only property?

Question

I'm having trouble getting the new System.Text.Json to deserialize collections stored on read-only properties.

Consider these classes:

public class SomeItem {
    public string Label { get; set; }
}

public class SomeObjectWithItems {

    public string Label { get; set; }

    // Note this property is read-only but the collection it points to is read/write
    public ObservableCollection<SomeItem> Items { get; }
        = new ObservableCollection<SomeItem>();
}

Here's the JSON:

{
  "Label": "First Set",
  "Items": [
    {
      "Label": "Item 1"
    },
    {
      "Label": "Item 2"
    },
    {
      "Label": "Item 3"
    },
    {
      "Label": "Item 4"
    }
  ]
}

Here's the code I'm running...

var json = ...;
var obj = JsonSerializer.Deserialize<SomeObjectWithItems>(json);
Debug.WriteLine($"Item Count for '{obj.Label}': {obj.Items.Count}");  

The above outputs the following:

Item Count for 'First Set': 0

If I change Items to be read/write, then it works, but so many of our models have read-only properties that hold mutable collections so I'm wondering if we can even use this.

Note: Json.NET handles this correctly, internally calling the 'Add' method on the existing collection rather than creating a new one, but I don't know how to achieve that outside of writing custom converters for all the classes we have defined.

Answer 1

This is by design for collections that don't have a setter. To avoid issues with adding to pre-populated collections (that the serializer doesn't instantiate) the deserializer uses "replace" semantics which requires the collection to have a setter.

Source: https://github.com/dotnet/corefx/issues/41433

There was an open issue for "Support adding to collections if no setter", discussion and design moved to "Support modifying already initialized properties and fields when deserializing JSON" (planned for .NET 8).

My recommendation is continue to use Json.NET in this case unless you want to write a custom converter.

https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-converters-how-to?pivots=dotnet-5-0

Custom converter from GitHub, not tested this myself:

class MagicConverter : JsonConverterFactory
{
    public override bool CanConvert(Type typeToConvert) =>
        !typeToConvert.IsAbstract &&
        typeToConvert.GetConstructor(Type.EmptyTypes) != null &&
        typeToConvert
            .GetProperties()
            .Where(x => !x.CanWrite)
            .Where(x => x.PropertyType.IsGenericType)
            .Select(x => new
            {
                Property = x,
                CollectionInterface = x.PropertyType.GetGenericInterfaces(typeof(ICollection<>)).FirstOrDefault()
            })
            .Where(x => x.CollectionInterface != null)
            .Any();

    public override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options) => (JsonConverter)Activator.CreateInstance(typeof(SuperMagicConverter<>).MakeGenericType(typeToConvert))!;

    class SuperMagicConverter<T> : JsonConverter<T> where T : new()
    {
        readonly Dictionary<string, (Type PropertyType, Action<T, object>? Setter, Action<T, object>? Adder)> PropertyHandlers;
        public SuperMagicConverter()
        {
            PropertyHandlers = typeof(T)
                .GetProperties()
                .Select(x => new
                {
                    Property = x,
                    CollectionInterface = !x.CanWrite && x.PropertyType.IsGenericType ? x.PropertyType.GetGenericInterfaces(typeof(ICollection<>)).FirstOrDefault() : null
                })
                .Select(x =>
                {
                    var tParam = Expression.Parameter(typeof(T));
                    var objParam = Expression.Parameter(typeof(object));
                    Action<T, object>? setter = null;
                    Action<T, object>? adder = null;
                    Type? propertyType = null;
                    if (x.Property.CanWrite)
                    {
                        propertyType = x.Property.PropertyType;
                        setter = Expression.Lambda<Action<T, object>>(
                            Expression.Assign(
                                Expression.Property(tParam, x.Property),
                                Expression.Convert(objParam, propertyType)),
                            tParam,
                            objParam)
                            .Compile();
                    }
                    else
                    {
                        if (x.CollectionInterface != null)
                        {
                            propertyType = x.CollectionInterface.GetGenericArguments()[0];
                            adder = Expression.Lambda<Action<T, object>>(
                                Expression.Call(
                                    Expression.Property(tParam, x.Property),
                                    x.CollectionInterface.GetMethod("Add"),
                                    Expression.Convert(objParam, propertyType)),
                                tParam,
                                objParam)
                                .Compile();
                        }
                    }
                    return new
                    {
                        x.Property.Name,
                        setter,
                        adder,
                        propertyType
                    };
                })
                .Where(x => x.propertyType != null)
                .ToDictionary(x => x.Name, x => (x.propertyType!, x.setter, x.adder));
        }
        public override void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options) => throw new NotImplementedException();
        public override T Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            var item = new T();
            while (reader.Read())
            {
                if (reader.TokenType == JsonTokenType.EndObject)
                {
                    break;
                }
                if (reader.TokenType == JsonTokenType.PropertyName)
                {
                    if (PropertyHandlers.TryGetValue(reader.GetString(), out var handler))
                    {
                        if (!reader.Read())
                        {
                            throw new JsonException($"Bad JSON");
                        }
                        if (handler.Setter != null)
                        {
                            handler.Setter(item, JsonSerializer.Deserialize(ref reader, handler.PropertyType, options));
                        }
                        else
                        {
                            if (reader.TokenType == JsonTokenType.StartArray)
                            {
                                while (true)
                                {
                                    if (!reader.Read())
                                    {
                                        throw new JsonException($"Bad JSON");
                                    }
                                    if (reader.TokenType == JsonTokenType.EndArray)
                                    {
                                        break;
                                    }
                                    handler.Adder!(item, JsonSerializer.Deserialize(ref reader, handler.PropertyType, options));
                                }
                            }
                            else
                            {
                                reader.Skip();
                            }
                        }
                    }
                    else
                    {
                        reader.Skip();
                    }
                }
            }
            return item;
        }
    }
}

Usage:

var options = new JsonSerializerOptions { Converters = { new MagicConverter() } };

var adsfsdf = System.Text.Json.JsonSerializer.Deserialize<Grrrr>("{\"Meow\":[1,2,3]}", options);
var adsfsdf2 = System.Text.Json.JsonSerializer.Deserialize<Grrrr>("{\"Meow\":null}", options);
var adsfsdf3 = System.Text.Json.JsonSerializer.Deserialize<Grrrr>("{\"Meow\":[1,2,3],\"Rawr\":\"asdf\"}", options);
var adsfsdf4 = System.Text.Json.JsonSerializer.Deserialize<Grrrr>("{\"Meow\":[1,2,3],\"Rawr\":null}", options);
var adsfsdf5 = System.Text.Json.JsonSerializer.Deserialize<Grrrr>("{\"Meow\":[1,2,3],\"Rawr\":\"asdf\",\"SubGrr\":{\"Meow\":[1,2,3],\"Rawr\":\"asdf\"}}", options);

Source:

https://github.com/dotnet/runtime/issues/30258#issuecomment-564847072

Answer 2

In .NET 7 and later, Microsoft has added the ability to programmatically customize the serialization contract that System.Text.Json creates for each .NET type. Using this API you could add a typeInfo modifier to add synthetic setters to read-only Collection<T> properties that populate the collection returned by the read-only property with the deserialized collection.

To do this, define the following modifier methods:

public static partial class JsonExtensions
{
    public static Action<JsonTypeInfo> CreateReadOnlyCollectionPropertySetters(Type type) => typeInfo => 
    {
        if (!type.IsAssignableFrom(typeInfo.Type))
            return;
        CreateReadOnlyCollectionPropertySetters(typeInfo);
    };

    public static void CreateReadOnlyCollectionPropertySetters(JsonTypeInfo typeInfo)
    {
        if (typeInfo.Kind != JsonTypeInfoKind.Object)
            return;
        // Add synthetic collection setters.
        foreach (var property in typeInfo.Properties)
        {
            if (property.Get != null && property.Set == null && property.PropertyType.GetCollectionItemType() is {} itemType)
            {
                var method = typeof(JsonExtensions).GetMethod(nameof(JsonExtensions.CreateGetOnlyCollectionPropertySetter),
                                                              BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static)!;
                var genericMethod = method.MakeGenericMethod(new[] { itemType });
                var setter = genericMethod.Invoke(null, new object[] { property }) as Action<object, object?>;
                property.Set = setter;
            }
        }
    }
    
    static Action<Object,Object?>? CreateGetOnlyCollectionPropertySetter<TItem>(JsonPropertyInfo property)
    {
        if (property.Get == null)
            return null;
        (var getter, var name) = (property.Get, property.Name);
        return (obj, value) =>
        {
            var oldValue = (ICollection<TItem>?)getter(obj);
            var newValue = value as ICollection<TItem>;
            if (newValue == oldValue)
                return;
            else if (oldValue == null || oldValue.IsReadOnly)
                throw new JsonException("Cannot populate list ${name} in ${obj}.");
            oldValue.Clear();
            if (newValue != null)
                foreach (var item in newValue)
                    oldValue.Add(item);
        };
    }

    public static IEnumerable<Type> BaseTypesAndSelf(this Type type)
    {
        while (type != null)
        {
            yield return type;
            type = type.BaseType!;
        }
    }

    static readonly Dictionary<Type, Type> InterfaceToOpenConcreteTypes = new ()
        {
            [typeof(IEnumerable<>)] = typeof(List<>),
            [typeof(ICollection<>)] = typeof(List<>),
            [typeof(IList<>)] = typeof(List<>),
            [typeof(ISet<>)] = typeof(HashSet<>),
        };
    
    static readonly HashSet<Type> MutableCollectionBaseTypes = new []
        {
            typeof(List<>),
            typeof(Collection<>),
            typeof(HashSet<>),
            typeof(SortedSet<>),
            // Add others as required
        }
        .ToHashSet();
    
    static Type? GetCollectionItemType(this Type type) 
    {
        if (type.IsArray || type.IsPrimitive)
            return null;
        else if (type.IsInterface)
        {
            return type.IsGenericType && InterfaceToOpenConcreteTypes.TryGetValue(type.GetGenericTypeDefinition(), out var _)
                ? type.GetGenericArguments()[0] : null;
        }
        else
        {
            return type.BaseTypesAndSelf()
                .Where(t => t.IsGenericType && MutableCollectionBaseTypes.Contains(t.GetGenericTypeDefinition()))
                .FirstOrDefault()?.GetGenericArguments()[0];
        }
    }
}

Then to deserialize your SomeObjectWithItems type, add do the following:

var options = new JsonSerializerOptions
{
    TypeInfoResolver = new DefaultJsonTypeInfoResolver
    {
        Modifiers = { 
            JsonExtensions.CreateReadOnlyCollectionPropertySetters(typeof(SomeObjectWithItems)), 
        },
    }
};

var obj = JsonSerializer.Deserialize<SomeObjectWithItems>(json, options);
Console.WriteLine($"Item Count for '{obj?.Label}': {obj?.Items.Count}");

Notes:

• JsonExtensions.CreateReadOnlyCollectionPropertySetters(Type type) adds synthetic setters for the specified type to read-only properties assignable to one of the types List<T>, Collection<T>, HashSet<T> or SortedSet<T>, or declared as IList<T>, ICollection<T>, IEnumerable<T>, or ISet<T>.

  To add additional collection base types, modify JsonExtensions.MutableCollectionBaseTypes.

• To add synthetic setters to all types, use the modifier JsonExtensions.CreateReadOnlyCollectionPropertySetters.

• It should be possible to introduce a similar modifier for read-only Dictionary<TKey, TValue> properties.

• The modifier assumes that the read-only collection property is already allocated by the constructor at the time it is deserialized. If not, an exception is thrown.

Demo fiddle here.