How can I generate this property implementation using Expressions instead of emitting IL?

Question

I'm trying to generate classes at runtime that implement property getters with a body that calls a method on the generated class's base class. Here's an example of a simple interface, along with a hand-written implementation that I'm trying to duplicate and the base class.

public interface IGenerated : IBase { decimal Property1 { get; } }

public class GeneratedByHand : ImplBase<IGenerated> { 
    public decimal Property1 { get { return Get(s => s.Property1); } }
}

public interface IBase { string _KeyPrefix { get; set; } }

public abstract class ImplBase<T> : IBase
    where T : IBase
{
    public virtual string _KeyPrefix { get; set; }

    protected virtual TResult Get<TResult>(Expression<Func<T, TResult>> property) { 
        return GetValue<TResult>(GetPropertyName(property)); 
    }

    private string GetPropertyName<TResult>(Expression<Func<T, TResult>> property) { 
        return ""; // reflection stuff to get name from property expression goes here
    }
    private TResult GetValue<TResult>(string keyPart) { 
        return default(TResult); // does something like: return ReallyGetValue<TResult>(_KeyPrefix + keyPart);
    }
}

I have a working implementation of the generator that emits IL to build the method, but if I can do it with Expressions I think that will be easier to expand and maintain. I will need to look for custom attributes on the property definitions and use that to call different method overloads on the base class in the property implementations.

Here's where I've gotten building an expression for the property get implementation. What I don't really understand is building the Call expression, if I'm setting it up correctly to do the equivalent of this.Get() or base.Get(). Right now it throws a System.ArgumentException : Invalid argument value Parameter name: method at CompileToMethod

public void CreateExpressionForGetMethod(MethodBuilder getBuilder, Type interfaceType, Type baseType, PropertyInfo property, MethodInfo getMethod)
{
    var settingsParam = Expression.Parameter(interfaceType, "s");
    var propGetterExpr = Expression.Property(settingsParam, property);

    var propGetterExprFuncType = typeof(Func<,>).MakeGenericType(interfaceType, property.PropertyType);
    var propGetterLambda = Expression.Lambda(propGetterExprFuncType, propGetterExpr, settingsParam);

    var baseGetMethodInfo = 
        baseType.GetMethods(BindingFlags.Instance | BindingFlags.NonPublic)
        .Where(m => {
            var parameters = m.GetParameters();
            return m.Name == "Get" &&
                    parameters != null && parameters.Count() == 1 && parameters[0].ParameterType != typeof(string);
        })
        .First().MakeGenericMethod(property.PropertyType);

    var getExprType = typeof(Expression<>).MakeGenericType(propGetterExprFuncType);
    var getExprParam = Expression.Parameter(getExprType, "expression");

    var getCallExpr = Expression.Call(Expression.Parameter(baseType, "inst"), baseGetMethodInfo, propGetterLambda);

    var getFuncType = typeof(Func<,>).MakeGenericType(getExprType, property.PropertyType);
    var propLambda = Expression.Lambda(getFuncType, getCallExpr, getExprParam);
    propLambda.CompileToMethod(getBuilder);
}

I'm not really sure where to go from here. I've tried a few other variations of arguments to Expression.Call, but everything else had Call throwing exceptions for the parameters being the wrong types.

Here's a buildable version of all the sample code I'm working with, including the working IL emitter:

using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
using System.Reflection.Emit;
using NUnit.Framework;



namespace ExpressionGenerationTest
{
    [TestFixture]
    public class UnitTests
    {
        [Test]
        public void CreateAndSaveAssembly()
        {
            var implGenerator = new ImplBuilder();
            var generatedType = implGenerator.CreateImplementation(typeof(IGenerated));
            implGenerator.SaveAssembly();
        }
    }


    public interface IBase { string _KeyPrefix { get; set; } }


    public abstract class ImplBase<T> : IBase
        where T : IBase
    {
        public virtual string _KeyPrefix { get; set; }

        protected virtual TResult Get<TResult>(Expression<Func<T, TResult>> property) { return GetValue<TResult>(GetPropertyName(property)); }

        private string GetPropertyName<TResult>(Expression<Func<T, TResult>> property) { return ""; } // reflection stuff to get name from property expression goes here
        private TResult GetValue<TResult>(string keyPart) { return default(TResult); } // does something like: return ReallyGetValue(_KeyPrefix + keyPart);
    }


    public interface IGenerated : IBase { decimal Property1 { get; } }

    public class GeneratedByHand : ImplBase<IGenerated> { public decimal Property1 { get { return Get(s => s.Property1); } } }



    public class ImplBuilder
    {
        private const string _assemblyNameBase = "ExpressionGenerationTest.Impl";

        public static ImplBuilder Default { get { return _default.Value; } }
        private static readonly Lazy<ImplBuilder> _default = new Lazy<ImplBuilder>(() => new ImplBuilder());

        private ConcurrentDictionary<Type, Type> _types = new ConcurrentDictionary<Type, Type>();
        private AssemblyBuilder _assemblyBuilder = null;
        private volatile ModuleBuilder _moduleBuilder = null;
        private object _lock = new object();

        private void EnsureInitialized()
        {
            if (_moduleBuilder == null) {
                lock (_lock) {
                    if (_moduleBuilder == null) {
                        _assemblyBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(new AssemblyName(_assemblyNameBase), AssemblyBuilderAccess.RunAndSave);
                        _moduleBuilder = _assemblyBuilder.DefineDynamicModule(_assemblyBuilder.GetName().Name, _assemblyNameBase + ".dll");
                    }
                }
            }
        }

        public void SaveAssembly() { _assemblyBuilder.Save(_assemblyNameBase + ".dll"); }
        public TSettings CreateInstance<TSettings>() { return (TSettings)Activator.CreateInstance(_types.GetOrAdd(typeof(TSettings), CreateImplementation)); }
        public void CreateImplementations(IEnumerable<Type> types) { foreach (var t in types) _types.GetOrAdd(t, InternalCreateImplementation); }
        public Type CreateImplementation(Type interfaceType) { return _types.GetOrAdd(interfaceType, InternalCreateImplementation); }
        private Type InternalCreateImplementation(Type interfaceType)
        {
            EnsureInitialized();

            var baseType = typeof (ImplBase<>).MakeGenericType(interfaceType);
            var typeBuilder = _moduleBuilder.DefineType(
                (interfaceType.IsInterface && interfaceType.Name.StartsWith("I") 
                    ? interfaceType.Name.Substring(1) 
                    : interfaceType.Name) + "Impl",
                TypeAttributes.Public | TypeAttributes.Class |
                TypeAttributes.AutoClass | TypeAttributes.AnsiClass |
                TypeAttributes.BeforeFieldInit | TypeAttributes.AutoLayout,
                baseType,
                new [] {interfaceType});

            foreach (var p in GetPublicProperties(interfaceType).Where(pi => pi.DeclaringType != typeof(IBase))) {
                var iGet = p.GetGetMethod();
                if (iGet != null) {
                    var getBuilder =
                        typeBuilder.DefineMethod(iGet.Name,
                            MethodAttributes.Public | MethodAttributes.Virtual | MethodAttributes.SpecialName | MethodAttributes.HideBySig,
                            p.PropertyType, Type.EmptyTypes);

                    //EmitILForGetMethod(getBuilder, interfaceType, baseType, p, iGet);
                    CreateExpressionForGetMethod(getBuilder, interfaceType, baseType, p, iGet);
                    typeBuilder.DefineMethodOverride(getBuilder, iGet);
                }
            }

            var implementationType = typeBuilder.CreateType();
            return implementationType;
        }


        public void CreateExpressionForGetMethod(MethodBuilder getBuilder, Type interfaceType, Type baseType, PropertyInfo property, MethodInfo getMethod)
        {
            var settingsParam = Expression.Parameter(interfaceType, "s");
            var propGetterExpr = Expression.Property(settingsParam, property);

            var propGetterExprFuncType = typeof(Func<,>).MakeGenericType(interfaceType, property.PropertyType);
            var propGetterLambda = Expression.Lambda(propGetterExprFuncType, propGetterExpr, settingsParam);

            var baseGetMethodInfo = 
                baseType.GetMethods(BindingFlags.Instance | BindingFlags.NonPublic)
                .Where(m => {
                    var parameters = m.GetParameters();
                    return m.Name == "Get" &&
                            parameters != null && parameters.Count() == 1 && parameters[0].ParameterType != typeof(string);
                })
                .First().MakeGenericMethod(property.PropertyType);

            var getExprType = typeof(Expression<>).MakeGenericType(propGetterExprFuncType);
            var getExprParam = Expression.Parameter(getExprType, "expression");

            var getCallExpr = Expression.Call(Expression.Parameter(baseType, "inst"), baseGetMethodInfo, propGetterLambda);

            var getFuncType = typeof(Func<,>).MakeGenericType(getExprType, property.PropertyType);
            var propLambda = Expression.Lambda(getFuncType, getCallExpr, getExprParam);
            propLambda.CompileToMethod(getBuilder);
        }


        public void EmitILForGetMethod(MethodBuilder getBuilder, Type interfaceType, Type baseType, PropertyInfo property, MethodInfo getMethod)
        {
            var getGen = getBuilder.GetILGenerator();
            var retVal = getGen.DeclareLocal(property.PropertyType);
            var expParam = getGen.DeclareLocal(typeof(ParameterExpression));
            var expParams = getGen.DeclareLocal(typeof(ParameterExpression[]));
            getGen.Emit(OpCodes.Ldarg_0);
            getGen.Emit(OpCodes.Ldtoken, interfaceType);
            getGen.Emit(OpCodes.Call, typeof(Type).GetMethod("GetTypeFromHandle"));
            getGen.Emit(OpCodes.Ldstr, "s");
            getGen.Emit(OpCodes.Call, typeof(Expression).GetMethod("Parameter", new [] {typeof(Type), typeof(string)}));
            getGen.Emit(OpCodes.Stloc, expParam);
            getGen.Emit(OpCodes.Ldloc, expParam);
            getGen.Emit(OpCodes.Ldtoken, getMethod);
            getGen.Emit(OpCodes.Call, typeof(MethodBase).GetMethod("GetMethodFromHandle", new [] {typeof(RuntimeMethodHandle)}, null));
            getGen.Emit(OpCodes.Castclass, typeof(MethodInfo));
            getGen.Emit(OpCodes.Call, typeof(Expression).GetMethod("Property", new[] {typeof(Expression), typeof(MethodInfo)}));
            getGen.Emit(OpCodes.Ldc_I4_1);
            getGen.Emit(OpCodes.Newarr, typeof(ParameterExpression));
            getGen.Emit(OpCodes.Stloc, expParams);
            getGen.Emit(OpCodes.Ldloc, expParams);
            getGen.Emit(OpCodes.Ldc_I4_0);
            getGen.Emit(OpCodes.Ldloc, expParam);
            getGen.Emit(OpCodes.Stelem_Ref);
            getGen.Emit(OpCodes.Ldloc, expParams);

            var lambdaMethodInfo = 
                typeof(Expression).GetMethods(BindingFlags.Public | BindingFlags.Static)
                .Where(x => { 
                    var parameters = x.GetParameters();
                    return x.Name == "Lambda" && 
                            x.IsGenericMethodDefinition &&
                            parameters.Count() == 2 &&
                            parameters[0].ParameterType == typeof(Expression) &&
                            parameters[1].ParameterType == typeof(ParameterExpression[]);
                }).FirstOrDefault();

            var lambdaFuncType = typeof(Func<,>);
            lambdaFuncType = lambdaFuncType.MakeGenericType(interfaceType, property.PropertyType);
            lambdaMethodInfo = lambdaMethodInfo.MakeGenericMethod(lambdaFuncType);

            getGen.Emit(OpCodes.Call, lambdaMethodInfo);


            var baseGetMethodInfo = 
                baseType.GetMethods(BindingFlags.Instance | BindingFlags.NonPublic)
                .Where(m => {
                    var parameters = m.GetParameters();
                    return m.Name == "Get" &&
                            parameters != null && parameters.Count() == 1 && parameters[0].ParameterType != typeof(string);
                }).FirstOrDefault();

            baseGetMethodInfo = baseGetMethodInfo.MakeGenericMethod(property.PropertyType);

            getGen.Emit(OpCodes.Callvirt, baseGetMethodInfo);

            getGen.Emit(OpCodes.Stloc_0);
            var endOfMethod = getGen.DefineLabel();
            getGen.Emit(OpCodes.Br_S, endOfMethod);
            getGen.MarkLabel(endOfMethod);
            getGen.Emit(OpCodes.Ldloc_0);
            getGen.Emit(OpCodes.Ret);
        }


        // from http://stackoverflow.com/a/2444090/224087
        public static PropertyInfo[] GetPublicProperties(Type type)
        {
            if (!type.IsInterface)
                return type.GetProperties(BindingFlags.FlattenHierarchy | BindingFlags.Public | BindingFlags.Instance);

            var propertyInfos = new List<PropertyInfo>();
            var considered = new List<Type>();
            var queue = new Queue<Type>();

            considered.Add(type);
            queue.Enqueue(type);
            while (queue.Count > 0) {
                var subType = queue.Dequeue();

                foreach (var subInterface in subType.GetInterfaces()) {
                    if (considered.Contains(subInterface)) 
                        continue;

                    considered.Add(subInterface);
                    queue.Enqueue(subInterface);
                }

                var typeProperties = subType.GetProperties(BindingFlags.FlattenHierarchy | BindingFlags.Public | BindingFlags.Instance);
                var newPropertyInfos = typeProperties.Where(x => !propertyInfos.Contains(x));
                propertyInfos.InsertRange(0, newPropertyInfos);
            }

            return propertyInfos.ToArray();
        }
    }
}

Just a note, if not using `Expression<...>`, it is normally easier just to use `ILGenerator`. So `CreateExpressionForGetMethod` is kind of pointless IMO. — leppie, Apr 11 '14 at 18:14
@leppie when I compare CreateExpressionForGetMethod & EmitILForGetMethod, the former appears simpler to write and understand. — quentin-starin, Apr 11 '14 at 18:17
@SK-logic perhaps not as much as I was thinking. Both have been difficult to debug and tricky to get correct. At least with IL I'm basically just copying the compiler. — quentin-starin, Apr 11 '14 at 18:19
I am implying that you are using Expressions the hard way. Let the compiler do the job. — leppie, Apr 11 '14 at 18:30
@leppie: There's no nice lambda syntax for creating expressions for unknown (variable) property access. — Ben Voigt, Apr 11 '14 at 18:31
The IL bits actually use `Expression` ... That is just silly. You should be using reflection. /cc @BenVoigt — leppie, Apr 11 '14 at 18:38
@leppie Thinking about your comment, I actually should have the generated implementation call `Get(string)`. `Get(Expression)` is convenience to keep string constants out of manual implementations. Can't believe I over looked that. Guess I like to make things complicated. — quentin-starin, Apr 11 '14 at 18:48
@leppie: Yeah, that is surprising... but if you look at the manually written version of the method, it contains a lambda. Naively imitating the MSIL generated by the compiler leads to the code shown. — Ben Voigt, Apr 11 '14 at 18:49
@BenVoigt: Classes are the poor man's closures. There are sufficient classes available in .NET (or just write one if needed) to emulate a closure. But generally, letting the compiler create those (sadly very inefficient) structures is the way to go. — leppie, Apr 11 '14 at 18:53
It's generally beneficial to have a compiler at hand for such small errands, so once you have something rich enough you'll reuse it over and over again anyway. Unfortunately, neither Expressions, nor a managed compiler (mono or roslyn) are usable enough, they won't let you build custom ASTs, so building your own generic compiler is still required, but it's pretty easy to do for IL. — SK-logic, Apr 11 '14 at 19:49

Ben Voigt · Answer 1 · 2014-04-11T20:59:30.257

What I don't really understand is building the Call expression, if I'm setting it up correctly to do the equivalent of this.Get() or base.Get().

If you are calling a virtual method, then this.Get(), which accesses the most-derived override (which could even be defined in a descendant of the current class), uses the callvirt instruction. And it doesn't matter what type you reflect against to get the MethodInfo, because they all share the same virtual table slot.

To emit base.Get(), you must

use the call instruction
reflect against the base class type

Because callvirt does some extra things besides v-table lookup, including a null pointer check, the C# compiler uses it for all virtual and non-virtual calls, except those involving the base keyword.

In particular, anonymous delegates and lambdas can't make use of the base keyword, since only descendant types can make non-virtual calls to virtual methods (at least in verifiable code), and the lambda is actually hosted by a closure type.

So unfortunately for your use case, there's no way to express a base call using lambda notation or expression trees. Expression.CompileToMethod only generates callvirt. Well, that isn't exactly correct. It generates call for calls to static methods and instance methods of value types. But instance methods of reference types use only callvirt. You can see this in System.Linq.Expressions.Compiler.LambdaCompiler.UseVirtual

Thanks @hvd for confirming this based on comments found in the Microsoft Reference Source for UseVirtual

I'm not having trouble with the IL part. That's works perfectly. I'm trying to do it with Expressions instead. — quentin-starin, Apr 11 '14 at 18:16
You could make `UseVirtual` link to [the MS reference source](http://referencesource.microsoft.com/System.Core/R/23f9e61cd9599c03.html). The comment there explicitly states "expression trees do not support "base.Foo()" style calling." — , Apr 11 '14 at 20:46

How can I generate this property implementation using Expressions instead of emitting IL?

1 Answers1