1

Currently I'm working on a project which has an additional layer built on top of Entityframework for Converting Queries from entity to DTO. We use a custom queryables to abstract the expression creation to call convert before materializing.

public Expression ConvertExpression(Expression origExpression)
{
    var expandedExpression = ExpressionExtensions.Expand(origExpression);
    var dummyValueExpr = expandedExpression as ConstantExpression;
    if (null != dummyValueExpr)
    {
        // Check if the expression is actually a dummy marker for some wrapper queryable
        var dummyQueryableValueConstant = dummyValueExpr.Value as IDummyQueryableValueConstant;
        if (null == dummyQueryableValueConstant)
        {
            // It's possible that the constant value is also another ConvertedQueryable.
            var innerQueryable = dummyValueExpr.Value as IConvertedQueryable;
            if ((null != innerQueryable) && (innerQueryable != this) && (innerQueryable.Expression is ConstantExpression))
            {
                return innerQueryable.ConvertExpression(innerQueryable.Expression);
            }
        }
        if (null != dummyQueryableValueConstant)
        {
            return this.ConvertExpression(dummyQueryableValueConstant.StoredQueryable.Expression);
        }
    }
    return expandedExpression;
}

We've overridden IQueryable Provider methods to call our conversion before execution:

public override object Execute(Expression expression)
{
    var constructors = AddConstructorsFromExpression(expression);
    var convertedExpression = this.ExpressionConverter.Visit(expression);
    var result = this._queryProvider.Execute(convertedExpression);
    if (null == result)
    {
        return null;
    }
    var types = GetInnerAndOuterTypesForInnerObject(result, convertedExpression.Type, expression.Type);

    return this._innerToOuterConverterFactory()
                   .Convert(result, types.Key, types.Value, x => GetConstructorOrNull(constructors, x));
}

The Include statements work fine when I am not using a GroupBy or Select(x => new { x }); My first intuition was that the expression Generated By the queryable was wrong. So I mapped the expressions based on using our context vs using entityframework on the northwind

.Call System.Linq.Queryable.GroupBy( .Call System.Linq.Queryable.Where( .Call (.Call .Constant1[StagingNorthWindTest.Customer]>(System.Data.Entity.Core.Objects.ObjectQuery1[StagingNorthWindTest.Customer]).MergeAs(.Constant(AppendOnly)) ).IncludeSpan(.Constant(System.Data.Entity.Core.Objects.Span)), '(.Lambda #Lambda1))

.Lambda #Lambda1(StagingNorthWindTest.Customer $x) { True }

.Lambda #Lambda2(StagingNorthWindTest.Customer $x) { $x.City }

.Call System.Linq.Queryable.GroupBy( .Call System.Linq.Queryable.Where( .Call (.Call .Constant1[BDBPayroll.Services.DataAccess.RowEntities.DivisionRptPackageRow]>(System.Data.Entity.Core.Objects.ObjectQuery1[BDBPayroll.Services.DataAccess.RowEntities.DivisionRptPackageRow]).MergeAs(.Constant(AppendOnly)) ).IncludeSpan(.Constant(System.Data.Entity.Core.Objects.Span)), '(.Lambda #Lambda1))

.Lambda #Lambda1(BDBPayroll.Services.DataAccess.RowEntities.DivisionRptPackageRow $x) { True }

.Lambda #Lambda2(BDBPayroll.Services.DataAccess.RowEntities.DivisionRptPackageRow $x) { $x.DivisionRptPackageID }

The expressions Generated for the same actions against northwind vs our context use the same syntax and pathing. However the internal DbQueryProvider seems to ignore my include spans despite them properly containing includes.

Is there anything special that entityframework does when generating Projections I.E Group by or anonymous Selects?

EDIT

I finally found out a small difference when I use the normal EF, the include calls the dbQuery Path while, when I use my context it calls the QueryableExtensions.CommonInclude 

  if (dbQuery != null)
    return (IQueryable<T>) dbQuery.Include(path);
  ObjectQuery<T> objectQuery = source as ObjectQuery<T>;
  if (objectQuery != null)
    return (IQueryable<T>) objectQuery.Include(path);
  return QueryableExtensions.CommonInclude<IQueryable<T>>(source, path);

What is the difference between these two methods?

johnny 5
  • 19,893
  • 50
  • 121
  • 195
  • Are you querying an MS SQL database? – Travis J Feb 15 '18 at 19:56
  • Yeah this is for MS Sql – johnny 5 Feb 15 '18 at 19:57
  • Is it possible that the type for your TSource for the Include statement changes using the expression conversion? In other words, if you were using `.Include(car -> car.Engine)` and you switch to a different base type using your expression conversion, then perhaps that would negate the include statement. – Travis J Feb 15 '18 at 20:00
  • I was thinking that, originally but our top level expressions are just dummies that hold the internal queryable. All the statements that affect the queryable are being applied on the base query. These queries work fine if I don’t call group by or if I don’t select into an anonymous. This whole day I’ve been digging into entityframework looking for the code which converts the includes but I haven’t found it yet – johnny 5 Feb 15 '18 at 20:04
  • I have a feeling that it’s something weird like that include statement associated with the expression is expected to be in a certain location relative to the queryable but gets moved during expression conversion – johnny 5 Feb 15 '18 at 20:06
  • Alright, well based on that I think the prime suspect here is the projection through grouping or selecting which is causing the issue. You already state this in general, but I would posit that the how is more related to the formation of the expression than anything that is happening in Entity Framework. I think you should try to ensure that the type is maintained instead of using an anonymous type or a plain object. Also, looking in EF you won't find that, you need to look into the query provider: System.Data.SqlClient – Travis J Feb 15 '18 at 20:08
  • re: moved. That is also possible. If the Include does not come first in the tree evaluation, then it will definitely have issues. – Travis J Feb 15 '18 at 20:09
  • Thanks, I'll check out the SqlClient, I just spent half the day trying to figure out how to get EF working so I could walk it, because apparently microsoft doesn't want to post that on their symbol server. Interesting so the include has to come first in the tree from the debug views it looks fine, but I'll take a deeper look into it – johnny 5 Feb 15 '18 at 20:24
  • @TravisJ Do you happen to know the difference between the DbQuery Include vs the CommonInclude? – johnny 5 Feb 15 '18 at 20:37
  • I was only aware of https://msdn.microsoft.com/en-us/library/bb738708(v=vs.110).aspx – Travis J Feb 15 '18 at 20:47
  • Ah, hm apparently there is a difference. The one I linked prior was an older version, and is the ObjectQuery (presumably from older EF's where Include was primarily taking a string). However, the one I use and thought that I had linked to is the DbExtensions Include from EF6, https://msdn.microsoft.com/en-us/library/gg671236(v=vs.103).aspx. The difference between these, or the ones related to your use? I would assume none, but perhaps the way they process a string versus an `Expression>` is the difference and that is part of the issue you are finding? Unsure. – Travis J Feb 15 '18 at 21:34
  • @Travis the CommonInclude seems to try to invoke the include when it is converting the SQL based off of the type IQueryable. The DbQuery Include seems to return a new DbQuery and sets the include directly on the internal set. I have a feeling my issue is related to this but I'm not sure, It's hard to invoke this with out rewriting half of the custom context were using – johnny 5 Feb 15 '18 at 21:40
  • What is the boxed type of `source` in `QueryableExtensions.CommonInclude>(source, path)`? Is it `IQueryable` or something else (like perhaps `DbSet`)? – Travis J Feb 15 '18 at 22:09
  • The source is a DbSet but its stored in a IQueryable, because at conversion time we only track our TopLevelEntity and not the Internal Row. This invokes the IQueryable Include and not the IQueryable include, after testing this doesn't seem to be the issue, I think I just need to hop into the SQLQuery Generator and walk the expression. To see what happening. – johnny 5 Feb 15 '18 at 22:50

0 Answers0