Can I clone an IQueryable to run on a DbSet for another DbContext?

Question

Suppose I have built up, through some conditional logic over many steps, an IQueryable<T> instance we'll call query.

I want to get a count of total records and a page of data, so I want to call query.CountAsync() and query.Skip(0).Take(10).ToListAsync(). I cannot call these in succession, because a race condition occurs where they both try to run a query on the same DbContext at the same time. This is not allowed:

"A second operation started on this context before a previous asynchronous operation completed. Use 'await' to ensure that any asynchronous operations have completed before calling another method on this context. Any instance members are not guaranteed to be thread safe."

I do not want to 'await' the first before even starting the second. I want to fire off both queries as soon as possible. The only way to do this is to run them from separate DbContexts. It seems ridiculous that I might have to build the entire query (or 2, or 3) side-by-side starting with different instances of DbSet. Is there any way to clone or alter an IQueryable<T> (not necessarily that interface, but it's underlying implementation) such that I can have one copy that runs on DbContext "A", and another that will run on DbContext "B", so that both queries can be executing simultaneously? I'm just trying to avoid recomposing the query X times from scratch just to run it on X contexts.

score 5 · Answer 1 · answered Jun 28 '18 at 17:28

There is no standard way of doing that. The problem is that EF6 query expression trees contain constant nodes holding ObjectQuery instances which are bound to the DbContext (actually the underlying ObjectContext) used when creating the query. Also there is a runtime check before executing the query if there are such expressions bound to a different context than the one executing the query.

The only idea that comes in my mind is to process the query expression tree with ExpressionVisitor and replace these ObjectQuery instances with new ones bound to the new context.

Here is a possible implementation of the aforementioned idea:

using System.Data.Entity.Core.Objects;
using System.Data.Entity.Infrastructure;
using System.Linq;
using System.Linq.Expressions;

namespace System.Data.Entity
{
    public static class DbQueryExtensions
    {
        public static IQueryable<T> BindTo<T>(this IQueryable<T> source, DbContext target)
        {
            var binder = new DbContextBinder(target);
            var expression = binder.Visit(source.Expression);
            var provider = binder.TargetProvider;
            return provider != null ? provider.CreateQuery<T>(expression) : source;
        }

        class DbContextBinder : ExpressionVisitor
        {
            ObjectContext targetObjectContext;
            public IQueryProvider TargetProvider { get; private set; }
            public DbContextBinder(DbContext target)
            {
                targetObjectContext = ((IObjectContextAdapter)target).ObjectContext;
            }
            protected override Expression VisitConstant(ConstantExpression node)
            {
                if (node.Value is ObjectQuery objectQuery && objectQuery.Context != targetObjectContext)
                    return Expression.Constant(CreateObjectQuery((dynamic)objectQuery));
                return base.VisitConstant(node);
            }
            ObjectQuery<T> CreateObjectQuery<T>(ObjectQuery<T> source)
            {
                var parameters = source.Parameters
                    .Select(p => new ObjectParameter(p.Name, p.ParameterType) { Value = p.Value })
                    .ToArray();
                var query = targetObjectContext.CreateQuery<T>(source.CommandText, parameters);
                query.MergeOption = source.MergeOption;
                query.Streaming = source.Streaming;
                query.EnablePlanCaching = source.EnablePlanCaching;
                if (TargetProvider == null)
                    TargetProvider = ((IQueryable)query).Provider;
                return query;
            }
        }
    }
}

One difference with the standard EF6 LINQ queries is that this produces ObjectQuery<T> rather than DbQuery<T>, although except that ToString() does not return the generated SQL, I haven't noticed any difference in the further query building / execution. It seems to work, but use it with care and on your own risk.

score 4 · Answer 2 · 2018-06-29T16:52:07.173

You could write a function to build up your query, taking DbContext as a parameter.

 public IQueryable<T> MyQuery(DbContext<T> db)
 {
     return db.Table
              .Where(p => p.reallycomplex)
              ....
              ...
              .OrderBy(p => p.manythings);
 }

I've done this many times and it works well.
Now it's easy to make queries with two different contexts:

IQueryable<T> q1 = MyQuery(dbContext1);
IQueryable<T> q2 = MyQuery(dbContext2);

If your concern was the execution time taken to build the IQueryable objects, then my only suggestion is don't worry about it.

score 3 · Answer 3 · answered Jul 02 '18 at 11:20

So you have an IQueryable<T> that will be performed on DbContext A as soon as the query is executed and you want the same query to run on DbContext B when the query is executed.

For this you'll have to understand the difference between an IEnumerable<T> and an IQueryable<T>.

An IEnumerable<T> holds all code to enumerate over the elements that the enumerable represents. The enumeration starts when GetEnumerator and MoveNext are called. This can be done explicitly. However it is usually done implicitly by functions like foreach, ToList, FirstOrDefault, etc.

An IQueryable does not hold the code to enumerate, it holds an Expression and a Provider. The Provider knows who will execute the query, and it knows how to translate the Expression into the language that is understood by the query executioner.

Due to this separation, it is possible to let the same Expression be executed by different data sources. They don't even have to be of the same type: one data source can be a database management system that understands SQL, the other one could be a comma separated file.

As long as you concatenate Linq statements that return an IQueryable, the query is not executed, only the Expression is changed.

As soon as enumeration starts, either by calling GetEnumerator / MoveNext, or by using foreach or one of the LINQ functions that do not return an IQueryable, the Provider will translate the Expression into the language the the data source understands and communicates with the data source to execute the query. The result of the query is an IEnumerable, which can be enumerated as if all data was in local code.

Some Providers are smart and use some buffering, so that not all data is transferred to local memory, but only part of the data. New data is queried when needed. So if you do a foreach in a database with a zillion elements, only the first few (thousands) elements are queried. More data is queried if your foreach runs out of fetched data.

So you already have one IQueryable<T>, therefore you have an Expression a Provider and an ElementType. You want the same Expression / ElementType to be executed by a differentProvider. You even want to change theExpression` slightly before you execute it.

Therefore you need to be able to create an object that implements IQueryable<T> and you want to be able to set the Expression, ElementType and a Provider

class MyQueryable<T> : IQueryable<T>
{
     public type ElementType {get; set;}
     public Expression Expression {get; set;}
     public Provider Provider {get; set;}
} 

IQueryable<T> queryOnDbContextA= dbCotextA ...
IQueryable<T> setInDbContextB = dbContextB.Set<T>();

IQueryable<T> queryOnDbContextB = new MyQueryable<T>()
{
     ElementType = queryOnDbContextA.ElementType,
     Expression = queryOnDbContextB.Expression,
     Provider = setInDbContextB.Provider,
}

If desired you can adjust the query on the other context before executing it:

var getPageOnContextB = queryOnDbContextB
    .Skip(...)
    .Take(...);

Both queries are still not executed yet. Execute them:

var countA = await queryOnContextA.CountAsync();
var fetchedPageContextB = await getPageOnContextB.ToListAsync();

Can I clone an IQueryable to run on a DbSet for another DbContext?

3 Answers3