1

I have a problem where adding a child object to a parent object is exceptionally slow when it should not be. There are tens of thousands of child objects (33k records in this case), but none of those are children of the parent object in question.

When I add the first child to the parent it takes more than one minute to complete:

public class ParentEntity // POCO generated by EF TT4 templates
{
    public virtual int Id { get; set; }
    public virtual ICollection<ChildEntity> ChildEntities  {}
} 

public class ChildEntity // POCO generated by EF TT4 templates
{
    public virtual int Id { get; set; }
    public virtual int ParentEntityId { get; set; }
    public virtual ParentEntity ParentEntity { get; set; }
    public virtual Warehouse Warehouse  { get; set; }
    public virtual WarehouseLocation  WarehouseLocation  { get; set; }
}

public class Warehouse { // etc } // another POCO class
public class WarehouseLocation { // etc } // another POCO class

// somewhere in a controller action method...
var parent = _parentEntityService.GetBy(id);
var child = new ChildEntity{ ParentEntityId = id, 
                             WarehouseId = id2, WarehouseLocationId = id3 };

// ChildEntities.Add() takes more than one minute to add the 
// first and only child to this parent
// why would this be so incredibly slow?

parent.ChildEntities.Add(child);

What is the best way to approach finding a speed problem in EntityFramework?

Update: EFProf shows that it issues three SQL queries:

SELECT * FROM ChildEntities where ParentId = id
SELECT * FROM ChildEntities where WarehouseId = id2
SELECT * FROM ChildEntities where WarehouseLocation = id3

Why does it load these for every single ChildEntity, when it should just load them for the current child only?

Edit 2: As per @LadislavMrnka the extra queries are caused by the template's Fixup method. But when I comment out those methods and comment out the call to Fixup it is still slow. Is this not the correct way to remove the fixup (it looks like it is removed to me):

public class ChildEntity {
public virtual Warehouse Warehouse
{
    get { return _warehouse; }
    set
    {
        if (!ReferenceEquals(_warehouse, value))
        {
            var previousValue = _warehouse;
            _warehouse = value;
            //FixupWarehouse(previousValue); // commented out
        }
    }
}
JK.
  • 21,477
  • 35
  • 135
  • 214
  • Try changing like this, `var parent = _parentEntityService.GetBy(id); var child = new ChildEntity{ Foo = "", Bar = "", Parent=parent };_entityService.Save(child)` – Jayantha Lal Sirisena Jan 03 '12 at 03:35
  • And use EF Profiler (http://efprof.com/) to see the inside of entity framework. – Jayantha Lal Sirisena Jan 03 '12 at 03:37
  • Does that `parent` entity have large number of `ChildEntities`? – Eranga Jan 03 '12 at 03:37
  • @Jayantha I've updated the details, we already use `var child = new ChildEntity{ Foo = "", Bar = "", ParentId=parentId }`. And the slowness occurrs on the ChildEntities.Add() call, before any saving has been done. I will download the trial of EFProf now. – JK. Jan 03 '12 at 03:41
  • @Eranga no the parent entity has zero child entities – JK. Jan 03 '12 at 03:42
  • 1
    If you are using `var child = new ChildEntity{ Foo = "", Bar = "", ParentId=parentId }` you don't need to use `ChildEntities.Add()` method. – Jayantha Lal Sirisena Jan 03 '12 at 03:43
  • @Jayantha yeah that's true but not in this case :) If we don't use ChildEntities.Add() then we cannot enumerate the children without first committing and then requerying. – JK. Jan 03 '12 at 03:47
  • 2
    have you tried profiling with sql server profiler to see what is happening – Sam Saffron Jan 03 '12 at 03:49
  • @Sam I'm trying to get EF profiler working but it doesnt work well with EFCachingProvider, so have to disable that first. – JK. Jan 03 '12 at 04:05
  • @JK. sql server profiler will require no code changes ... assuming you are on sql server – Sam Saffron Jan 03 '12 at 04:14
  • Ok EF prof running - OMG I 'm going to buy this right now. Anyway .. EFprfo shows just one sql statement was run: `SELECT [Extent1].[Id] AS [Id], .. etc for all fields FROM [dbo].[ChildEntities] AS [Extent1]`. And this one SQL timed out after getting 23k rows. BUT I think I see the problem: the debugger stopped while loading a child of ChildEntity. It looks like it has taken every single child entity and tried to fully load the object graph for each (and ChildEntity has children 6 levels deep). Why does it try to load so much? I have lazy loading turned ON. – JK. Jan 03 '12 at 04:34
  • Updated question with more details learnt by using EFProf. The slow speed is because it loads two child properties of every single ChildEntity instead of only for the current child – JK. Jan 03 '12 at 04:47
  • You are not showing all code. What is warehouse and warehouse location and how is it related to child. Do you have any custom code in your entities? – Ladislav Mrnka Jan 03 '12 at 08:31
  • @LadislavMrnka they are just navigation properties created by the template because ChildEntity has a FK relationship to Warehouse and WarehouseLocation (yes that's bad design and should be child.Warehouse.Location, its on our list to fix). There is no custom code in the template - what code would you like to see? – JK. Jan 03 '12 at 10:13
  • @JK.: Did you also try `Parent` instead of `ParentId` like Jayantha suggested? – Merlyn Morgan-Graham Jan 03 '12 at 10:22
  • Did you also somewhere set `Warehouse` or `WarehouseLocation` into your new child? – Ladislav Mrnka Jan 03 '12 at 10:25
  • @LadislavMrnka yes when creating the child entity `var child = new ChildEntity{ Foo = "", Bar = "", ParentEntityId = id, WarehouseId = 1, WarehouseLocationId =1 }; ` – JK. Jan 03 '12 at 11:00
  • I took 15 comments to get all information needed for answering your question. – Ladislav Mrnka Jan 03 '12 at 11:24

2 Answers2

5

This is your problem:

var child = new ChildEntity 
               { 
                   Foo = "", 
                   Bar = "", 
                   ParentEntityId = id, 
                   WarehouseId = 1, 
                   WarehouseLocationId = 1 
               };

parent.ChildEntities.Add(child);

IMHO it is all about fixup collections hidden in code generated by POCO template. Fixup + lazy loading = performance problems. Fixup tries to make everything in your model in sync. It means that if you set one side of navigation property or FK property it will try to make sure that navigation property on opposite side of relation reflects the change as well. The problem is that if the navigation property is not loaded it will trigger lazy loading. In your case it looks like setting Warehouse has first fixed up the navigation property in ChildEntity and after that tried to fixup navigation property on Warehouse instance but its child entities collection was not loaded => lazy loading causing

SELECT * FROM ChildEntities where WarehouseId = some id

The same happened in case of WarehouseLocation. The first query is result of adding child to not loaded collection on parent entity.

The solution is either to modify template and get rid of all fixups (for example DbContext POCO template for EFv4.1+ don't use fixups anymore) or turn off lazy loading for this operation by calling:

context.ContextOptions.LazyLoadingEnabled = false;

// Your insert logic here

context.ContextOptions.LazyLoadingEnalbed = true;

You can even wrap the code in custom IDisposable like:

public class DisableLazyLoadingScope : IDisposable
{
    private readonly ObjectContext context;

    public DisableLazyLoadingScope(ObjectContext context)
    {
        this.context = context;
        context.ContextOptions.LazyLoadingEnabled = false;
    }

    public void Dispose()
    {
        context.ContextOptions.LazyLoadingEnabled = true;
    }
}

And use it like:

using (new DisableLazyLoadingScope(context)
{
    // Your insert logic here
}
Ladislav Mrnka
  • 360,892
  • 59
  • 660
  • 670
  • Aha I understand, thanks. What benefit does Fixup give us? I only have it because it was in the TT4 template, and I would be happy to remove it if it isn't needed. I'm using EF4.1 now, but when the templates were first added I was using an earlier EF version. – JK. Jan 03 '12 at 11:59
  • Fixup keeps loaded data in sync. If you set `WarehouseId` on one side fixup will ensure that `Warehouse` navigation property has correct `Warehouse` entity and `Warehouse` entity has your child in its collection of related entities. The side effect is that if you are using lazy loading it will load those entities even if you don't want to use them. – Ladislav Mrnka Jan 03 '12 at 13:05
  • I've removed fixup from ChildEntity and it is still slow. I'm going to try removing fixup from all entities now. I looked at the 4.1 self tracking entity template and it still has Fixup (implemented differently): `var previousValue = _warehouse; _warehouse = value; FixupWarehouse(previousValue); OnNavigationPropertyChanged("Warehouse");` – JK. Jan 03 '12 at 22:45
  • Self tracking enity template is for EFv4 unless they released a new one and self tracking must do fixup. Start by turning off lazy loading to verify that it is the problem. – Ladislav Mrnka Jan 03 '12 at 22:47
  • I haven't been able to turn off lazy loading -> nothing is loaded when this is on. I cannot use .Include() because all queries are handled by `IQueryable GetQuery { return ObjectSet.AsQueryable(); }` inside of my GenericRepository. I inherited this app and it already had self tracking - I've just been reading and found very different views on whether STE should be used or not (for a straightfoward MVC site) – JK. Jan 03 '12 at 23:52
  • I've read more and seen lots of references to lazy loading not working with STE but they refer to older versions of EF eg http://stackoverflow.com/q/6571954/325727 and http://social.msdn.microsoft.com/Forums/en/adonetefx/thread/6133d1a8-5952-4168-bcb6-756202341238 etc. Actually I probably don't need self tracking entities - if we make a change to an entity we call.Commit() in the same method. What is the likely effect of removing them? Will I need to change how I commit or load or anything else? – JK. Jan 04 '12 at 00:10
  • Are you sure you are using STEs because indeed STEs don't support lazy loading. STEs make sense only if you want to have tracked entity passed by process boundary (loading and saving is each handled by different context instance). – Ladislav Mrnka Jan 04 '12 at 08:35
  • Disabling lazy loading made my parent/child relation load 10x faster. Thanks for this! – garethb Aug 12 '14 at 04:41
0

This helped me enormously:

context.Configuration.AutoDetectChangesEnabled = false;

I'm not sure if this goes for you too, but if I understand your question correctly it's slow for you when you just do the Add?

That will still allow you to insert objects, but EF will take much less effort in navigating the object hierarchy. I believe this was problematic when updating entities though, but I don't know for sure.

A second method (assuming you use DbContext) is:

using (var dbCtx = new MyDataContext())
{
  var ctx = ((IObjectContextAdapter)dbCtx).ObjectContext;

  var customers = ctx.CreateObjectSet<Customer>();

  customers.AddObject(customer);
}

Because ObjectContext handles things differently internally with regards to changes. I can't find the source on that right now, I'll try to find it later.

JulianR
  • 16,213
  • 5
  • 55
  • 85