Why multiple filters are applied even if query recreated on each iteration

Question

I found this code below in a file called Filter.cs in a project created with Microsoft App Studio. Although I am a veteran C# programmer, I'm short on experience with LINQ predicate expression builders. I can tell that the code below it is "meta-logic" for flexibly building a query given a list of filter predicates containing type field info and a set of data values to inject into the sub-expressions. What I can't figure out is how the "expression" variable in the following statement:

query = query.Where(expression).AsQueryable()" 

.. is concatenating the per-field expressions into a more complex query expression that is finally executed at the end of the code to create the ObservableCollection result. If it was "query +=" I could infer a chaining action like an Event handler field, but as a straight assignment statement it baffles me since I would expect it to replace the last value the expression variable got from the last loop iteration, thereby resetting it in the process and losing its previous value(s). What is going on here?

public class Filter<T>
{
    public static ObservableCollection<T> FilterCollection(
        FilterSpecification filter, IEnumerable<T> data)
    {
        IQueryable<T> query = data.AsQueryable();               
        foreach (var predicate in filter.Predicates)
        {
            Func<T, bool> expression;
            var predicateAux = predicate;
            switch (predicate.Operator)
            {
                case ColumnOperatorEnum.Contains:
                    expression = x => predicateAux.GetFieldValue(x).ToLower().Contains(predicateAux.Value.ToString().ToLower());
                    break;
                case ColumnOperatorEnum.StartsWith:
                    expression = x => predicateAux.GetFieldValue(x).ToLower().StartsWith(predicateAux.Value.ToString().ToLower());
                    break;
                case ColumnOperatorEnum.GreaterThan:
                    expression = x => String.Compare(predicateAux.GetFieldValue(x).ToLower(), predicateAux.Value.ToString().ToLower(), StringComparison.Ordinal) > 0;
                    break;
                case ColumnOperatorEnum.LessThan:
                    expression = x => String.Compare(predicateAux.GetFieldValue(x).ToLower(), predicateAux.Value.ToString().ToLower(), StringComparison.Ordinal) < 0;
                    break;
                case ColumnOperatorEnum.NotEquals:
                    expression = x => !predicateAux.GetFieldValue(x).Equals(predicateAux.Value.ToString(), StringComparison.InvariantCultureIgnoreCase);
                    break;
                default:
                    expression = x => predicateAux.GetFieldValue(x).Equals(predicateAux.Value.ToString(), StringComparison.InvariantCultureIgnoreCase);
                    break;
            }

            // Why doesn't this assignment wipe out the expression function value from the last loop iteration?
            query = query.Where(expression).AsQueryable();
        }
        return new ObservableCollection<T>(query);
    }

Answer 1

My understanding is that you are having trouble understanding why this line executed in a loop

query = query.Where(expression).AsQueryable();

produces an effect similar to "concatenation" of expressions. A short answer is that it is similar to why

str = str + suffix;

produces a longer string, even though it is an assignment.

A longer answer is that the loop is building an expression one predicate at a time, and appends a Where to the sequence of conditions. Even though it is an assignment, it is built from the previous state of the object, so the previous expression is not "lost", because it is used as a base of a bigger, more complex, filter expression.

To understand it better, imagine that the individual expressions produced by the switch statement are placed into an array of IQueryable objects, instead of being appended to query. Once the array of parts is built, you would be able to do this:

var query = data.AsQueryable()
    .Where(parts[0]).AsQueryable()
    .Where(parts[1]).AsQueryable()
    ...
    .Where(parts[N]).AsQueryable();

Now observe that each parts[i] is used only once; after that, it is no longer needed. That is why you can build the chain of expressions incrementally in a loop: after the first iteration, query contains a chain that includes the first term; after the second iteration, it contains two first terms, and so on.

Answer 2

It doesn't "wipe it out" since it is chaining. It's handling it by assigning back to query. It's effectively like writing:

var queryTmp = query;
query = queryTmp.Where(expression).AsQueryable();

Each time you call .Where(expression).AsQueryable(), a new IQueryable<T> is being returned, and set to query. This IQueryable<T> is the result of the last .Where call. This means you effectively get a query that looks like:

query.Where(expression1).AsQueryable().Where(expression2).AsQueryable()...

Answer 3

Code essentially generates sequence of Where/AsQueryable calls. Not sure why you expect each loop to append expressions.

Essentially result is

query = query
  .Where(expression0).AsQueryable()
  .Where(expression1).AsQueryable()
  .Where(expression2).AsQueryable()

where I think you expect more like

query = query
  .Where(v => expression0(v) && expression1(v) && expression2(v) ...).AsQueryable()

Answer 4

The query variable name is a bit misleading. This code doesn't build up a long filter in the expression variable and then run it against the data set - it runs each filter against the data set, one at a time, until all of the filters have been run. The query variable just contains everything from the data that is left over from the previously run filters.

So this line:

query = query.Where(expression).AsQueryable();

is applying the filter to the existing data stored in query, and then saving the new (filtered) result back into the variable. The value of expression is overwritten each time through the loop, but we don't need it anymore because the filter has already been applied.