Newtonsoft JSON PreserveReferencesHandling with custom Equals usage

Question

I'm currently having some issues with Newtonsoft Json.

What I want is simple: Compare the Object which will be serialized with all Properties and Subproperties for Equality.

I tried now to Create my own EqualityComparer but it only compared with the Properties of the Parent Object.

Also, I tried to write my own ReferenceResolver but had no luck with it.

Let's talk with an Example:

public class EntityA
{
    int Foo {get; set;}

    public override bool Equals(object obj)
    {
        return (obj is EntityA other) && other.Foo == this.Foo;
    }
}

public class EntityB
{
    int Bar {get; set;}

    EntityA Parent {get; set;}

    public override bool Equals(object obj)
    {
        return (obj is EntityB other) && other.Bar == this.Bar;
    }
}

public class InnerWrapper
{
    public string FooBar {get; set;}

    public EntityB BEntity {get; set;}
}

public class OuterClass
{
    public EntityA AEntity { get; set;}

    List<InnerWrapper> InnerElements {get; set;}
}    

Now what I want is to have the References from EntityB to EntityA. They are in my case always the same. So what I expect is, that in the JSON in every EntityB the reference to EntityA is written as ref. The Equal of the Entities overwrites the Equals to check if they are the same. They are Database Objects, so that they are equals as soon as their ID is the same. In this case I've called them Foo and Bar.

What I've tried is as following:

public class MyEqualComparer : IEqualityComparer
{
    public bool Equals(object x, object y)
    {
        return x.Equals(y);
    }

    public int GetHashCode(object obj)
    {
        return obj.GetHashCode();
    }
}

with the following JSON Settings

public static readonly JsonSerializerSettings JsonSerializerSettings = new JsonSerializerSettings
{
    TypeNameHandling = TypeNameHandling.All,
    NullValueHandling = NullValueHandling.Ignore,
    FloatParseHandling = FloatParseHandling.Decimal,
    Formatting = Formatting.Indented,
    PreserveReferencesHandling = PreserveReferencesHandling.Objects,
    EqualityComparer = new MyEqualComparer(),
    ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
    Error = (sender, args) => Log.Error(args.ErrorContext.Error, $"Error while (de)serializing: {args.ErrorContext}; object: {args.CurrentObject}")
};

But it doesn't work. It compares totally wrong values. For example the EntityA from the OuterClass with each of the InnerWrapper. But not with the Properties or even Subproperties (in this case the Properties of the EntityB of the InnerWrapper).

With a custom ReferenceResolver, I've also no luck, because the settings above are really generic and I don't have any idea how to write a generic one.

Do you have any idea how to get this work?

// Edit:

Below an example what I expect:

{
    "$id" : "1",
    "AEntity": {
        "$id": "2",
        "Foo": 200
    },
    "InnerElements": [
        {
            "$id": "3",
            "Bar": 20,
            "Parent": {
                "$ref" : "2"
            }
        },
        {
            "$id": "4",
            "Bar": 21,
            "Parent": {
                "$ref" : "2"
            }
        },
        {
            "$id": "5",
            "Bar": 23,
            "Parent": {
                "$ref" : "2"
            }
        },
        {
            "$id": "6",
            "Bar": 24,
            "Parent": {
                "$ref" : "2"
            }
        },
        {
            "$id": "7",
            "Bar": 25,
            "Parent": {
                "$ref" : "2"
            }
        }
    ]

}

And this is what I get:

    {
    "$id" : "1",
    "AEntity": {
        "$id": "2",
        "Foo": 200
    },
    "InnerElements": [
        {
            "$id": "3",
            "Bar": 20,
            "Parent": {
                "$id": "8",
                "Foo": 200
            }
        },
        {
            "$id": "4",
            "Bar": 21,
            "Parent": {
                "$id": "9",
                "Foo": 200
            }
        },
        {
            "$id": "5",
            "Bar": 23,
            "Parent": {
                "$id": "10",
                "Foo": 200
            }
        },
        {
            "$id": "6",
            "Bar": 24,
            "Parent": {
                "$id": "11",
                "Foo": 200
            }
        },
        {
            "$id": "7",
            "Bar": 25,
            "Parent": {
                "$id": "12",
                "Foo": 200
            }
        }
    ]

}

Of course, in this case, the impact is low. But my real scenario is much bigger.

Answer 1

As stated in this answer to JSON.NET Serialization - How does DefaultReferenceResolver compare equality? by Andrew Whitaker, Json.NET exclusively uses reference equality when preserving references via PreserveReferencesHandling. The setting JsonSerializerSettings.EqualityComparer is intended for reference loop detection rather than reference preservation and resolution, as explain in this answer to Why doesn't reference loop detection use reference equality?.

Andrew's answer gives an example of a custom IReferenceResolver that resolves references using object equality for objects of a certain type and assumes all serialized objects are of that type. What you would like to do is to use object equality only for certain types (EntityA and EntityB) and fall back on Json.NET's default reference resolver for all other types.

You can accomplish this via the decorator pattern, in which you wrap an instance of Json.NET's reference resolver in your own IReferenceResolver. Then implement whatever logic is necessary for the types that need their own custom equality comparison, and pass everything else on to the underlying default resolver.

Here is one that meets your requirements:

public class SelectiveValueEqualityReferenceResolver : EquivalencingReferenceResolver
{
    readonly Dictionary<Type, Dictionary<object, object>> representatives;

    public SelectiveValueEqualityReferenceResolver(IReferenceResolver defaultResolver, IEnumerable<Type> valueTypes)
        : base(defaultResolver)
    {
        if (valueTypes == null)
            throw new ArgumentNullException();
        representatives = valueTypes.ToDictionary(t => t, t => new Dictionary<object, object>());
    }

    protected override bool TryGetRepresentativeObject(object obj, out object representative)
    {
        var type = obj.GetType();
        Dictionary<object, object> typedItems;

        if (representatives.TryGetValue(type, out typedItems))
        {
            return typedItems.TryGetValue(obj, out representative);
        }
        return base.TryGetRepresentativeObject(obj, out representative);
    }

    protected override object GetOrAddRepresentativeObject(object obj)
    {
        var type = obj.GetType();
        Dictionary<object, object> typedItems;

        if (representatives.TryGetValue(type, out typedItems))
        {
            object representative;
            if (!typedItems.TryGetValue(obj, out representative))
                representative = (typedItems[obj] = obj);
            return representative;

        }
        return base.GetOrAddRepresentativeObject(obj);
    }
}

public abstract class EquivalencingReferenceResolver : IReferenceResolver
{
    readonly IReferenceResolver defaultResolver;

    public EquivalencingReferenceResolver(IReferenceResolver defaultResolver)
    {
        if (defaultResolver == null)
            throw new ArgumentNullException();
        this.defaultResolver = defaultResolver;
    }

    protected virtual bool TryGetRepresentativeObject(object obj, out object representative)
    {
        representative = obj;
        return true;
    }

    protected virtual object GetOrAddRepresentativeObject(object obj)
    {
        return obj;
    }

    #region IReferenceResolver Members

    public void AddReference(object context, string reference, object value)
    {
        var representative = GetOrAddRepresentativeObject(value);
        defaultResolver.AddReference(context, reference, representative);
    }

    public string GetReference(object context, object value)
    {
        var representative = GetOrAddRepresentativeObject(value);
        return defaultResolver.GetReference(context, representative);
    }

    public bool IsReferenced(object context, object value)
    {
        object representative;

        if (!TryGetRepresentativeObject(value, out representative))
            return false;
        return defaultResolver.IsReferenced(context, representative);
    }

    public object ResolveReference(object context, string reference)
    {
        return defaultResolver.ResolveReference(context, reference);
    }

    #endregion
}

Which you would then use as follows:

var settings = new JsonSerializerSettings
{
    //Commented out TypeNameHandling since the JSON in the question does not include type information
    //TypeNameHandling = TypeNameHandling.All,
    NullValueHandling = NullValueHandling.Ignore,
    FloatParseHandling = FloatParseHandling.Decimal,
    Formatting = Formatting.Indented,
    PreserveReferencesHandling = PreserveReferencesHandling.Objects,
    ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
    ReferenceResolverProvider = () => new SelectiveValueEqualityReferenceResolver(
        new JsonSerializer().ReferenceResolver, 
        new [] { typeof(EntityA), typeof(EntityB) }),
    Error = (sender, args) => Log.Error(args.ErrorContext.Error, $"Error while (de)serializing: {args.ErrorContext}; object: {args.CurrentObject}")
};

var outer = JsonConvert.DeserializeObject<OuterClass>(jsonString, settings);

var json2 = JsonConvert.SerializeObject(outer, settings);

Note that I had to make a variety of fixes to your types to make this work:

public static class EqualityHelper
{
    public static bool? EqualsQuickReject<T1, T2>(T1 item1, T2 item2) 
        where T1 : class
        where T2 : class
    {
        if ((object)item1 == (object)item2)
            return true;
        else if ((object)item1 == null || (object)item2 == null)
            return false;

        if (item1.GetType() != item2.GetType())
            return false;

        return null;
    }
}

public class EntityA : IEquatable<EntityA> //Fixed added IEquatable<T>
{
    public int Foo { get; set; } // FIXED made public

    public override bool Equals(object obj)
    {
        return Equals(obj as EntityA);
    }

    // Fixed added required GetHashCode() that is compatible with Equals()
    public override int GetHashCode()
    {
        return Foo.GetHashCode();
    }

    #region IEquatable<EntityA> Members

    public bool Equals(EntityA other)
    {
        // FIXED - ensure Equals is reflexive, symmetric and transitive even when dealing with derived types
        var initial = EqualityHelper.EqualsQuickReject(this, other);
        if (initial != null)
            return initial.Value;
        return this.Foo == other.Foo;
    }

    #endregion
}

public class EntityB : IEquatable<EntityB> //Fixed added IEquatable<T>
{
    public int Bar { get; set; } // FIXED made public

    public EntityA Parent { get; set; } // FIXED made public

    public override bool Equals(object obj)
    {
        return Equals(obj as EntityB);
    }

    // Fixed added required GetHashCode() that is compatible with Equals()
    public override int GetHashCode()
    {
        return Bar.GetHashCode();
    }

    #region IEquatable<EntityB> Members

    public bool Equals(EntityB other)
    {
        // FIXED - ensure Equals is reflexive, symmetric and transitive even when dealing with derived types
        var initial = EqualityHelper.EqualsQuickReject(this, other);
        if (initial != null)
            return initial.Value;
        return this.Bar == other.Bar;
    }

    #endregion
}

public class InnerWrapper
{
    public string FooBar { get; set; }

    public EntityB BEntity { get; set; }
}

public class OuterClass
{
    public EntityA AEntity { get; set; }

    public List<EntityB> InnerElements { get; set; }//FIXED -- made public and corrected type to be consistent with sample JSON
}

Notes:

• SelectiveValueEqualityReferenceResolver works as follows. When constructed it is given a default reference resolver and a list of types for which to use object equality. Then, when one of the IReferenceResolver methods is called, it checks to see whether the incoming object is of one of the custom types. If so, it checks to see whether it has already encountered an equivalent object of the same type using object equality. If so, passes that initial object on to the default reference resolver. If not, it caches the incoming object as a defining instance of object-equivalent objects, then passes the incoming object on to the default reference resolver.

• The above logic only works if the overridden object.Equals() is a proper equivalence relation -- i.e. reflexive, symmetric and transitive.

  In your code this was not guaranteed to be the case if EntityA or EntityB were ever subclassed. Thus I modified your Equals() methods to require that the incoming object be of the same type, rather than just a compatible type.

• When Equals() is overriden, it is also necessary to override GetHashCode() in a compatible manner, such that equal objects have equal hash codes.

  This was not done in your code so I added the necessary logic to EntityA and EntityB.

• Json.NET's DefaultReferenceResolver is internal so I had to use a slightly hacky method to create one, namely construct a temporary JsonSerializer and grab its ReferenceResolver.

• SelectiveValueEqualityReferenceResolver is not thread safe so a fresh serializer instance should be used in each thread.

• SelectiveValueEqualityReferenceResolver is designed to generate identical $id values for object-equal objects during serialization. It's not designed to merge equal objects with different $id values into reference-equal objects during deserialization. I think that could be added if required.

Answer 2

Thanks dbc for helping out.

Your code works nearly like that what I wanted. In that example, it worked really fine (sorry for the code issues).

If made a small adjustment to your code, to not only rely on specific types.

public class SelectiveValueEqualityReferenceResolver : EquivalencingReferenceResolver
{
      private readonly Dictionary<Type, Dictionary<object, object>> _representatives;

      public SelectiveValueEqualityReferenceResolver(IReferenceResolver defaultResolver)
          : base(defaultResolver)
      {
          this._representatives = new Dictionary<Type, Dictionary<object, object>>();
      }

      protected override bool TryGetRepresentativeObject(object obj, out object representative)
      {
          var type = obj.GetType();
          if (type.GetTypeInfo().IsClass && this._representatives.TryGetValue(type, out var typedItems))
              return typedItems.TryGetValue(obj, out representative);

          return base.TryGetRepresentativeObject(obj, out representative);
      }

      protected override object GetOrAddRepresentativeObject(object obj)
      {
          var type = obj.GetType();

          if (!type.GetTypeInfo().IsClass)
              return base.GetOrAddRepresentativeObject(obj);

          if (!this._representatives.TryGetValue(type, out var typedItems))
          {
              typedItems = new Dictionary<object, object>();
              this._representatives.Add(type, typedItems);
          }

          if (!typedItems.TryGetValue(obj, out var representative))
              representative = typedItems[obj] = obj;

          return representative;
      }
}

This one uses the default comparer for all classes. For all other (structs, etc.) it uses the default one.