70

Given this extremely simple model:

public class MyContext : BaseContext
{
    public DbSet<Foo> Foos { get; set; }
    public DbSet<Bar> Bars { get; set; }
}

public class Foo
{
    public int Id { get; set; }
    public int Data { get; set; }
    [Required]
    public virtual Bar Bar { get; set; }
}

public class Bar
{
    public int Id { get; set; }
}

The following program fails:

object id;
using (var context = new MyContext())
{
    var foo = new Foo { Bar = new Bar() };
    context.Foos.Add(foo);
    context.SaveChanges();
    id = foo.Id;
}
using (var context = new MyContext())
{
    var foo = context.Foos.Find(id);
    foo.Data = 2;
    context.SaveChanges(); //Crash here
}

With a DbEntityValidationException. The message found in EntityValidationErrors is The Bar field is required..

However, if I force loading of the Bar property by adding the following line before SaveChanges:

var bar = foo.Bar;

Everything works fine. This also works if I remove the [Required] attribute.

Is this really the expected behavior? Are there any workarounds (besides loading every single required reference every time I want to update an entity)

Diego Mijelshon
  • 52,548
  • 16
  • 116
  • 154
  • 1
    I just tripped against this yesterday so I can confirm your observations. Am looking for a work around. This seems very unfortunate. – Xhalent May 18 '11 at 10:03
  • 1
    It is not only problem of navigation property. I already complained about that on MSDN: http://social.msdn.microsoft.com/Forums/en-US/adodotnetentityframework/thread/cacc3156-6b5f-4412-a8b8-a1279d28a536/ – Ladislav Mrnka May 18 '11 at 10:29
  • 1
    To be honest, I think EF proxies are simply broken and dangerous because of all these problems around nullability. See the issue here: https://entityframework.codeplex.com/workitem/1571 There is also the issue of failing to set an unloaded reference to null (because it is already null/unloaded). Basically, proxies do not work in EF, even change-tracking ones exhibit the same behaviour. The situation is appalling and everyone has to write hackarounds to fix basic, everyday situations. – Rob Kent Oct 02 '14 at 08:34

8 Answers8

54

I found the following post that had an answer for the same problem:

The cause of this problem is that in RC and RTM validation no longer lazy loads any properties. The reason this change was made is because when saving a lot of entities at once that have lazy loaded properties validation would get them one by one potentially causing a lot of unexpected transactions and crippling performance.

The workaround is to explicitly load all validated properties before saving or validating by using .Include(), you can read more on how to do this here: http://blogs.msdn.com/b/adonet/archive/2011/01/31/using-dbcontext-in-ef-feature-ctp5-part-6-loading-related-entities.aspx

My take on this is that is a pretty crappy proxy implementation. While unnecesarily walking the object graph and retriveing lazy-loaded properties is naturally something to be avoided (but apparently overlooked in Microsoft's first incarnation of EF), you shouldn't have to need to go un-proxying a wrapper to validate that it exists. On second thoughts, I'm not sure why you need to go walking the object graph anyway, surely the change tracker of the ORM knows what objects require validation.

I'm not sure why the problem exists, but I'm sure I wouldn't be having this problem if I was using say, NHibernate.

My 'workaround' - What I've done is define the Required nature of the relationship in a EntityTypeConfiguration class, and removed the Required attribute. This should make it work fine. It means that you will not validate the relationship, but it will fail the update. Not an ideal result.

friism
  • 19,068
  • 5
  • 80
  • 116
Xhalent
  • 3,914
  • 22
  • 21
  • 5
    I ended up writing a generic `LoadAllReferences` method. I couldn't be more disappointed at EF. – Diego Mijelshon May 18 '11 at 15:12
  • 6
    Thanks for the answer. This has got to the the single most stupid bug I've seen in a long time. How did anyone ever think this could be acceptable for an ORM? – Dark Falcon Aug 17 '12 at 18:21
  • 2
    I am disappointed in learning this. Isn't another workaround to make all Required navigation properties non-lazy by removing virtual? – Carl G Jan 09 '13 at 23:26
  • 2
    @CarlG. If you make all references non-lazy then you end up retriveing a undetermined number of objects from the database, any number of which you actually do need for a particular unit of work. That's why lazy loading available. – Xhalent Feb 06 '13 at 05:50
  • 2
    yeah, surely the framework knows that the Bar field has NOT been changed and therefore does NOT need to be checked. EF is a joke. i wish I hadn't chosen it, it's too late to switch now but I'll never use it again. – Spongman Sep 13 '13 at 22:49
  • If you don't feel like using an `EntityTypeConfiguration` class, you can always explicitly declare foreign key properties and link to the relationship navigation properties via `ForeignKeyAttribute`. It doesn't solve the issue with a lack of validation though - still fails on update. – dahvyd Nov 22 '13 at 19:58
  • This extremely annoying bug is another aspect of EF's half-implemented proxies. See this issue as well: https://entityframework.codeplex.com/workitem/2074 – Rob Kent Oct 01 '14 at 15:58
  • What I don't understand is that in my case my Master entity has a child_id entry in its table, so EF knows it has a value. The problem seems to be the same with the 'set-to-null is not detected' on an unloaded lazy property; the proxy assumes an unloaded property is actually null. In my case the lazy property is a byte array which could be several MBs, and I definitely do not want to load that every time. If I take off the Required attribute, I get a different error because of the orphaned reference. Yuk! – Rob Kent Oct 01 '14 at 16:02
  • The problem is that when EF loads a proxy, it should also read the foreign key values so that during validation it knows that although those properties are null it is because they are unloaded, not because they are not set. The whole situation with proxies is appalling: they basically do not work and are not fit for purpose and it is not good enough for the EF team to tell us WHY proxies behave this way if they simply cannot be relied on. – Rob Kent Oct 02 '14 at 08:37
  • Thanks a lot for the workaround solution. I was using custom `EagerLoadRelatedEntities`-style solution, which I really hated. This just saved me tons of time and prevented lot of stupid bugs. – Tom Pažourek Jun 22 '16 at 08:32
44

Ok, here is the real answer =)

First a little explanation

if you have a property (like your Bar) noting a FK (ForeignKey), you can also have the corresponding FK field in your model so if we only need the FK and not the actual Bar we don't need it to go to the database:

[ForeignKey("BarId")]
public virtual Bar Bar { get; set; }
public int BarId { get; set; }

Now, to answer your question, what you can do to make the Bar as Required is to flag the BarId property as required, but not the Bar itself:

[ForeignKey("BarId")]
public virtual Bar Bar { get; set; }
[Required] //this makes the trick
public int BarId { get; set; }

this works like a charm =)

Mikael Dúi Bolinder
  • 2,080
  • 2
  • 19
  • 44
Diego Garber
  • 449
  • 3
  • 2
  • 1
    Nice answer (upvoted). My FKs are named the same as the properties, So I'd have to do `[Required, Column("Bar"), ForeignKey("Bar")] public int? BarId { get; set; }`, which is ugly, as I'd be essentially hacking my domain model to satisfy EF's oddities. – Diego Mijelshon May 26 '11 at 14:46
  • 2
    The problem with this is that when creating a new Foo() you need to set both the Bar and BarId properties, if you just set the Bar property then you'll fail the required validation on BarId. Also, BarId needs to be nullable for the required attribute to work. – Xhalent Jun 12 '11 at 08:14
  • This worked for me. I think the BarId should be nullable to reflect that Bar has not been set yet, in addition to the fact that I think [Required] is meaningless on scalar property. @Xhalent you can set the BarId in your Bar property. – Carl G Jan 09 '13 at 23:50
  • Thanks for this answer! I didn't need the [Required] attribute, but I had no ForeignKey (Id) in my model - now it works like a charm! (I'm using EF5) – kapsiR Mar 14 '13 at 08:02
  • 1
    But what if you delete Foo, it will not cascade delete to Bar. When you remove Foo from the context and SaveChanges, Bar is set to null before deletion, and you then get this error: "Invalid data encountered. A required relationship is missing. Examine StateEntries to determine the source of the constraint violation." But there is nothing in StateEntries to indicate the problem. – Rob Kent Oct 01 '14 at 16:06
9

Transparent workaround to ignore error on unloaded references

In your DbContext, override ValidateEntity method to remove validation error on references that are not loaded.

    private static bool IsReferenceAndNotLoaded(DbEntityEntry entry, string memberName)
    {
        var reference = entry.Member(memberName) as DbReferenceEntry;
        return reference != null && !reference.IsLoaded;
    }

    protected override DbEntityValidationResult ValidateEntity(DbEntityEntry entityEntry,
                                                 IDictionary<object, object> items)
    {
        var result = base.ValidateEntity(entityEntry, items);
        if (result.IsValid || entityEntry.State != EntityState.Modified)
        {
            return result;
        }
        return new DbEntityValidationResult(entityEntry,
            result.ValidationErrors
                  .Where(e => !IsReferenceAndNotLoaded(entityEntry, e.PropertyName)));
    }

Pros :

  • Transparent and will not crash when you use inheritance, complex types, doesn't require modification on your model...
  • Only when validation fails
  • No reflection
  • Iterates only on invalid unloaded references
  • No useless data loading
Guillaume
  • 12,824
  • 3
  • 40
  • 48
5

Here's a semi-acceptable work-around:

var errors = this.context.GetValidationErrors();
foreach (DbEntityValidationResult result in errors) {
    Type baseType = result.Entry.Entity.GetType().BaseType;
    foreach (PropertyInfo property in result.Entry.Entity.GetType().GetProperties()) {
        if (baseType.GetProperty(property.Name).GetCustomAttributes(typeof(RequiredAttribute), true).Any()) {
            property.GetValue(result.Entry.Entity, null);
        }
    }
}
Community
  • 1
  • 1
friism
  • 19,068
  • 5
  • 80
  • 116
  • 1
    Yup, that's more or less what I'm doing these days. I even created an [OSS project](https://github.com/diegose/EntityFramework.Diegose) and [Nuget package](http://nuget.org/packages/EntityFramework.Diegose) with this as a feature. – Diego Mijelshon Oct 13 '12 at 11:08
  • 1
    Does this code work with inheritance? I have three levels of inheritance and I get a null ref which I think is because property.Name does not belong to the base type. – Rob Kent Oct 02 '14 at 09:12
  • @RobKent I would certainly like to know too as I got the exact same problem as you. Anyone knows? – JensOlsen112 Dec 18 '14 at 20:42
4

If anyone wants a general approach to solve this problem, here you have a custom DbContext which finds out properties based on these constraints:

  • Lazy Load is ON.
  • Properties with virtual
  • Properties having any ValidationAttribute attribute.

After retrieving this list, on any SaveChanges in which have something to modify it will load all references and collections automatically avoiding any unexpected exception.

public abstract class ExtendedDbContext : DbContext
{
    public ExtendedDbContext(string nameOrConnectionString)
        : base(nameOrConnectionString)
    {
    }

    public ExtendedDbContext(DbConnection existingConnection, bool contextOwnsConnection)
        : base(existingConnection, contextOwnsConnection)
    {
    }

    public ExtendedDbContext(ObjectContext objectContext, bool dbContextOwnsObjectContext)
        : base(objectContext, dbContextOwnsObjectContext)
    {
    }

    public ExtendedDbContext(string nameOrConnectionString, DbCompiledModel model)
        : base(nameOrConnectionString, model)
    {
    }

    public ExtendedDbContext(DbConnection existingConnection, DbCompiledModel model, bool contextOwnsConnection)
        : base(existingConnection, model, contextOwnsConnection)
    {
    }

    #region Validation + Lazy Loading Hack

    /// <summary>
    /// Enumerator which identifies lazy loading types.
    /// </summary>
    private enum LazyEnum
    {
        COLLECTION,
        REFERENCE,
        PROPERTY,
        COMPLEX_PROPERTY
    }

    /// <summary>
    /// Defines a lazy load property
    /// </summary>
    private class LazyProperty
    {
        public string Name { get; private set; }
        public LazyEnum Type { get; private set; }

        public LazyProperty(string name, LazyEnum type)
        {
            this.Name = name;
            this.Type = type;
        }
    }

    /// <summary>
    /// Concurrenct dictinary which acts as a Cache.
    /// </summary>
    private ConcurrentDictionary<Type, IList<LazyProperty>> lazyPropertiesByType =
        new ConcurrentDictionary<Type, IList<LazyProperty>>();

    /// <summary>
    /// Obtiene por la caché y si no lo tuviese lo calcula, cachea y obtiene.
    /// </summary>
    private IList<LazyProperty> GetLazyProperties(Type entityType)
    {
        return
            lazyPropertiesByType.GetOrAdd(
                entityType,
                innerEntityType =>
                {
                    if (this.Configuration.LazyLoadingEnabled == false)
                        return new List<LazyProperty>();

                    return
                        innerEntityType
                            .GetProperties(BindingFlags.Public | BindingFlags.Instance)
                            .Where(pi => pi.CanRead)
                            .Where(pi => !(pi.GetIndexParameters().Length > 0))
                            .Where(pi => pi.GetGetMethod().IsVirtual)
                            .Where(pi => pi.GetCustomAttributes().Exists(attr => typeof(ValidationAttribute).IsAssignableFrom(attr.GetType())))
                            .Select(
                                pi =>
                                {
                                    Type propertyType = pi.PropertyType;
                                    if (propertyType.HasGenericInterface(typeof(ICollection<>)))
                                        return new LazyProperty(pi.Name, LazyEnum.COLLECTION);
                                    else if (propertyType.HasGenericInterface(typeof(IEntity<>)))
                                        return new LazyProperty(pi.Name, LazyEnum.REFERENCE);
                                    else
                                        return new LazyProperty(pi.Name, LazyEnum.PROPERTY);
                                }
                            )
                            .ToList();
                }
            );
    }

    #endregion

    #region DbContext

    public override int SaveChanges()
    {
        // Get all Modified entities
        var changedEntries =
            this
                .ChangeTracker
                .Entries()
                .Where(p => p.State == EntityState.Modified);

        foreach (var entry in changedEntries)
        {
            foreach (LazyProperty lazyProperty in GetLazyProperties(ObjectContext.GetObjectType(entry.Entity.GetType())))
            {
                switch (lazyProperty.Type)
                {
                    case LazyEnum.REFERENCE:
                        entry.Reference(lazyProperty.Name).Load();
                        break;
                    case LazyEnum.COLLECTION:
                        entry.Collection(lazyProperty.Name).Load();
                        break;
                }
            }
        }

        return base.SaveChanges();
    }

    #endregion
}

Where IEntity<T> is:

public interface IEntity<T>
{
    T Id { get; set; }
}

These extensions were used in this code:

public static bool HasGenericInterface(this Type input, Type genericType)
{
    return
        input
            .GetInterfaces()
            .Any(x => x.IsGenericType && x.GetGenericTypeDefinition() == genericType);
}

public static bool Exists<T>(this IEnumerable<T> source, Predicate<T> predicate)
{
    foreach (T item in source)
    {
        if (predicate(item))
            return true;
    }

    return false;
} 

Hope it helps,

2

I know it's a bit late... However, ill post this here. Since i too got horribly annoyed with this. Just tell EF to Include the required field.

Notice the SMALL change

using (var context = new MyContext())
{
    var foo = context.Foos.Include("Bar").Find(id);
    foo.Data = 2;
    context.SaveChanges(); //Crash here
}
Aiden Strydom
  • 1,198
  • 2
  • 14
  • 43
0

Since this is still a problem in EF 6.1.1 I thought I would provide another answer that may suit some people, depending on their exact model requirements. To summarize the issue:

  1. You need to use a proxy for lazy loading.

  2. The property you are lazy loading is marked Required.

  3. You want to modify and save the proxy without having to force-load the lazy references.

3 is not possible with the current EF proxies (either of them), which is a serious shortcoming in my opinion.

In my case the lazy property behaves like a value type so its value is provided when we add the entity and never changed. I can enforce this by making its setter protected and not providing a method to update it, that is, it must be created through a constructor, eg:

var myEntity = new MyEntity(myOtherEntity);

MyEntity has this property:

public virtual MyOtherEntity Other { get; protected set; }

So EF will not perform validation on this property but I can ensure it is not null in the constructor. That is one scenario.

Assuming you do not want to use the constructor in that way, you can still ensure validation using a custom attribute, such as:

[RequiredForAdd]
public virtual MyOtherEntity Other { get; set; }

The RequiredForAdd attribute is a custom attribute that inherits from Attribute not RequiredAttribute. It has no properties or methods apart from its base ones.

In my DB Context class I have a static constructor which finds all the properties with those attributes:

private static readonly List<Tuple<Type, string>> validateOnAddList = new List<Tuple<Type, string>>();

static MyContext()
{
    FindValidateOnAdd();
}

private static void FindValidateOnAdd()
{
    validateOnAddList.Clear();

    var modelType = typeof (MyEntity);
    var typeList = modelType.Assembly.GetExportedTypes()
        .Where(t => t.Namespace.NotNull().StartsWith(modelType.Namespace.NotNull()))
        .Where(t => t.IsClass && !t.IsAbstract);

    foreach (var type in typeList)
    {
        validateOnAddList.AddRange(type.GetProperties(BindingFlags.Public | BindingFlags.Instance)
            .Where(pi => pi.CanRead)
            .Where(pi => !(pi.GetIndexParameters().Length > 0))
            .Where(pi => pi.GetGetMethod().IsVirtual)
            .Where(pi => pi.GetCustomAttributes().Any(attr => attr is RequiredForAddAttribute))
            .Where(pi => pi.PropertyType.IsClass && pi.PropertyType != typeof (string))
            .Select(pi => new Tuple<Type, string>(type, pi.Name)));
    }
}

Now that we have a list of properties we need to check manually, we can override validation and manually validate them, adding any errors to the collection returned from the base validator:

protected override DbEntityValidationResult ValidateEntity(DbEntityEntry entityEntry, IDictionary<object, object> items)
{
    return CustomValidateEntity(entityEntry, items);
}

private DbEntityValidationResult CustomValidateEntity(DbEntityEntry entry, IDictionary<object, object> items)
{
    var type = ObjectContext.GetObjectType(entry.Entity.GetType());

    // Always use the default validator.    
    var result = base.ValidateEntity(entry, items);

    // In our case, we only wanted to validate on Add and our known properties.
    if (entry.State != EntityState.Added || !validateOnAddList.Any(t => t.Item1 == type))
        return result;

    var propertiesToCheck = validateOnAddList.Where(t => t.Item1 == type).Select(t => t.Item2);

    foreach (var name in propertiesToCheck)
    {
        var realProperty = type.GetProperty(name);
        var value = realProperty.GetValue(entry.Entity, null);
        if (value == null)
        {
            logger.ErrorFormat("Custom validation for RequiredForAdd attribute validation exception. {0}.{1} is null", type.Name, name);
            result.ValidationErrors.Add(new DbValidationError(name, string.Format("RequiredForAdd validation exception. {0}.{1} is required.", type.Name, name)));
        }
    }

    return result;
}

Note that I am only interested in validating for an Add; if you wanted to check during Modify as well, you would need to either do the force-load for the property or use a Sql command to check the foreign key value (shouldn't that already be somewhere in the context)?

Because the Required attribute has been removed, EF will create a nullable FK; to ensure you DB integrity you could alter the FKs manually in a Sql script that you run against your database after it has been created. This will at least catch the Modify with null issues.

Rob Kent
  • 5,183
  • 4
  • 33
  • 54
0

Just had the same problem in EF 6.1.2. To solve this your class should be like the following:

public class Foo {
    public int Id { get; set; }
    public int Data { get; set; }

    public int BarId { get; set; }

    public virtual Bar Bar { get; set; }

}

As you can see, the "Required" attribute is not needed, because the Bar property is already required since the BarId property is not nullable.

So if you wanted the Bar property to be nullable, you would have to write:

public class Foo {
    public int Id { get; set; }
    public int Data { get; set; }

    public int? BarId { get; set; }

    public virtual Bar Bar { get; set; }
}
André Lourenço
  • 668
  • 5
  • 10