Is it possible to serialize a C# code block?

Question

I'm using C# with .NET 3.5. Is it possible to serialize a block of code, transmit it somewhere, deserialize it, and then execute it?

An example usage of this would be:

Action<object> pauxPublish = delegate(object o)
{
    if (!(o is string))
    {
        return;
    }
    Console.WriteLine(o.ToString());
};
Transmitter.Send(pauxPublish);

With some remote program doing:

var action = Transmitter.Recieve();
action("hello world");

My end goal is to be able to execute arbitrary code in a different process (which has no prior knowledge of the code).

When working for a client who had to handle thousands of file imports each day from a couple of hundred source, I created a generic import tool where advanced users were allowed to define file columns and enter a line of C# to transform the input value to the appropriate type/format. Once the new definition was saved, we'd dynamically compile the import class for this source, then execute it when new files arrived. With 50 or so common cases auto-completable, it worked surprisingly well and saved an awful lot of change requests, as well as improving the import speed by an order of magnitude. — Basic, Aug 16 '15 at 04:35

score 20 · Accepted Answer · answered Dec 06 '08 at 13:54

20

YES!!!

We have done this for a very real case of performance. Doing this at runtime or using a DSL was not an option due to performance.

We compile the code into an assembly, and rip the IL out of the method. We then get all the metadata associated with this method and serialize the whole mess via XML, compress it, and put it in our database.

At re-hydration time, we re-constitute the IL with the metadata using the DynamicMethod class, and execute it.

We do this because of speed. We have thousands of little blocks of code. Unfortunately, to compile a block of code and run it on the fly takes at least 250 ms, which is way too slow for us. We took this approach, and it is working REALLY well. At run-time, it takes an unmeasurable amount of time to reconstitute the method and run it.

Only thing to keep an eye on... Signed assemblies and Unsigned assemblies cannot mix the serialized method data.

answered Dec 06 '08 at 13:54

Brian Genisio

47,787
16
124
167

This was the method I came up with and have been working on. The biggest problem seems to be converting the byte array that MethodBody gives into OpCodes to use with Reflection.Emit. I'm looking at http://weblogs.asp.net/rosherove/archive/2006/04/25/methodsvisualizer.aspx which might help. – ICR Dec 06 '08 at 14:12
o0o this sounds like the approach that I am looking for. If you have any more resources relating to think, links would be very much appreciated :) – NoizWaves Dec 06 '08 at 14:38
3

I can't make up my mind if what you have done is amazing or awful. Possibly both. – Anthony Dec 06 '08 at 14:54
http://blogs.msdn.com/haibo_luo/archive/2006/11/07/turn-methodinfo-to-dynamicmethod.aspx also seems pretty useful, perhaps more so. – ICR Dec 06 '08 at 16:24
Amazing or awful? I think it is a bit of both. I really can't do anything but think it is a hack of grand proportions... I was amazed when it finally worked... shocked by how evil/complex the solution really was. – Brian Genisio Dec 06 '08 at 18:59
We based our approach off of this article: http://www.codeproject.com/KB/cs/ExpressionEval.aspx . I should add that since we got it in place (with a TON of unit tests), we haven't had a single problem with the approach. – Brian Genisio Dec 06 '08 at 19:01

score 7 · Answer 2 · edited Apr 15 '12 at 21:45

7

You could try to use IronPython in your project. It's trivial to do what you are asking in Python. The Python code could call your C# methods. As for security, you could execute the code in a restricted environment of some kind (one example is RestrictedPython).

edited Apr 15 '12 at 21:45

Peter Mortensen

30,738
21
105
131

answered Dec 06 '08 at 10:33

Toni Ruža

7,462
2
28
31

score 4 · Answer 3 · answered Dec 06 '08 at 10:06

4

Generally speaking that sounds like a really bad idea and a big security hole.

You don't want another process to execute any code. Understand what you really need another process to do and build a little DSL around it.

answered Dec 06 '08 at 10:06

VVS

19,405
5
46
65

Absolutely, you cannot trust the caller you really aught to come up with a DSL. Take a look at MGrammar and Oslo. – justin.m.chase Dec 07 '08 at 06:13
Executing arbitrary code and/or storing it in database somewhere... what could possibly go wrong? Doesn't seem like an anti-pattern to me ;-) – Jacobs Data Solutions Jul 01 '10 at 12:27

score 3 · Answer 4 · edited Apr 11 '17 at 06:43

You could also send it as a string then use the CodeDomProvider to compile it, same result. I have an example bit of code thus:

using System;
using System.CodeDom.Compiler;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Text;
using Microsoft.CSharp;

namespace DynamicCodeApplication
{
    class azCodeCompiler
    {
        private List<string> assemblies;

        public azCodeCompiler()
        {
            assemblies = new List<string>();
            scanAndCacheAssemblies();
        }

        public Assembly BuildAssembly(string code)
        {

            CodeDomProvider prov = CodeDomProvider.CreateProvider("CSharp");
            string[] references = new string[] { };   // Intentionally empty, using csc.rsp
            CompilerParameters cp = new CompilerParameters(references)
                                        {
                                            GenerateExecutable = false,
                                            GenerateInMemory = true
                                        };
            string path = System.Runtime.InteropServices.RuntimeEnvironment.GetRuntimeDirectory();
            cp.CompilerOptions = "@" + path + @"\csc.rsp";
            CompilerResults cr = prov.CompileAssemblyFromSource(cp, code);

            foreach (CompilerError err in cr.Errors)
            {
                Console.WriteLine(err.ToString());
            }
            return cr.CompiledAssembly;
        }

        public object ExecuteCode(string code,
                                  string namespacename, string classname,
                                  string functionname, bool isstatic, params object[] args)
        {
            object returnval = null;
            Assembly asm = BuildAssembly(code);
            object instance = null;
            Type type = null;
            if (isstatic)
            {
                type = asm.GetType(namespacename + "." + classname);
            }
            else
            {
                instance = asm.CreateInstance(namespacename + "." + classname);
                type = instance.GetType();
            }
            MethodInfo method = type.GetMethod(functionname);
            returnval = method.Invoke(instance, args);
            return returnval;
        }

        private void scanAndCacheAssemblies()
        {

            /*
            foreach (string str in Directory.GetFiles(@"C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727"))
            {
                if (str.Contains(".dll"))
                {
                    foreach (string st in str.Split(new char[] { '\\' }))
                    {
                        if (st.Contains(".dll"))
                        {
                            assemblies.Add(st);
                        }
                    }
                }
            }
             * */

            assemblies.Add("Accessibility.dll");
            assemblies.Add("AspNetMMCExt.dll");
            assemblies.Add("cscompmgd.dll");
            assemblies.Add("CustomMarshalers.dll");
            assemblies.Add("IEExecRemote.dll");
            assemblies.Add("IEHost.dll");
            assemblies.Add("IIEHost.dll");
            assemblies.Add("Microsoft.Build.Conversion.dll");
            assemblies.Add("Microsoft.Build.Engine.dll");
            assemblies.Add("Microsoft.Build.Framework.dll");
            assemblies.Add("Microsoft.Build.Tasks.dll");
            assemblies.Add("Microsoft.Build.Utilities.dll");
            assemblies.Add("Microsoft.Build.VisualJSharp.dll");
            assemblies.Add("Microsoft.CompactFramework.Build.Tasks.dll");
            assemblies.Add("Microsoft.JScript.dll");
            assemblies.Add("Microsoft.VisualBasic.Compatibility.Data.dll");
            assemblies.Add("Microsoft.VisualBasic.Compatibility.dll");
            assemblies.Add("Microsoft.VisualBasic.dll");
            assemblies.Add("Microsoft.VisualBasic.Vsa.dll");
            assemblies.Add("Microsoft.Vsa.dll");
            assemblies.Add("Microsoft.Vsa.Vb.CodeDOMProcessor.dll");
            assemblies.Add("Microsoft_VsaVb.dll");
            assemblies.Add("mscorlib.dll");
            assemblies.Add("sysglobl.dll");
            assemblies.Add("System.configuration.dll");
            assemblies.Add("System.Configuration.Install.dll");
            assemblies.Add("System.Data.dll");
            assemblies.Add("System.Data.OracleClient.dll");
            assemblies.Add("System.Data.SqlXml.dll");
            assemblies.Add("System.Deployment.dll");
            assemblies.Add("System.Design.dll");
            assemblies.Add("System.DirectoryServices.dll");
            assemblies.Add("System.DirectoryServices.Protocols.dll");
            assemblies.Add("System.dll");
            assemblies.Add("System.Drawing.Design.dll");
            assemblies.Add("System.Drawing.dll");
            assemblies.Add("System.EnterpriseServices.dll");
            assemblies.Add("System.Management.dll");
            assemblies.Add("System.Messaging.dll");
            assemblies.Add("System.Runtime.Remoting.dll");
            assemblies.Add("System.Runtime.Serialization.Formatters.Soap.dll");
            assemblies.Add("System.Security.dll");
            assemblies.Add("System.ServiceProcess.dll");
            assemblies.Add("System.Transactions.dll");
            assemblies.Add("System.Web.dll");
            assemblies.Add("System.Web.Mobile.dll");
            assemblies.Add("System.Web.RegularExpressions.dll");
            assemblies.Add("System.Web.Services.dll");
            assemblies.Add("System.Windows.Forms.dll");
            assemblies.Add("System.XML.dll");
            assemblies.Add("vjscor.dll");
            assemblies.Add("vjsjbc.dll");
            assemblies.Add("vjslib.dll");
            assemblies.Add("vjslibcw.dll");
            assemblies.Add("vjssupuilib.dll");
            assemblies.Add("vjsvwaux.dll");
            assemblies.Add("vjswfc.dll");
            assemblies.Add("VJSWfcBrowserStubLib.dll");
            assemblies.Add("vjswfccw.dll");
            assemblies.Add("vjswfchtml.dll");
            assemblies.Add("Accessibility.dll");
            assemblies.Add("AspNetMMCExt.dll");
            assemblies.Add("cscompmgd.dll");
            assemblies.Add("CustomMarshalers.dll");
            assemblies.Add("IEExecRemote.dll");
            assemblies.Add("IEHost.dll");
            assemblies.Add("IIEHost.dll");
            assemblies.Add("Microsoft.Build.Conversion.dll");
            assemblies.Add("Microsoft.Build.Engine.dll");
            assemblies.Add("Microsoft.Build.Framework.dll");
            assemblies.Add("Microsoft.Build.Tasks.dll");
            assemblies.Add("Microsoft.Build.Utilities.dll");
            assemblies.Add("Microsoft.Build.VisualJSharp.dll");
            assemblies.Add("Microsoft.CompactFramework.Build.Tasks.dll");
            assemblies.Add("Microsoft.JScript.dll");
            assemblies.Add("Microsoft.VisualBasic.Compatibility.Data.dll");
            assemblies.Add("Microsoft.VisualBasic.Compatibility.dll");
            assemblies.Add("Microsoft.VisualBasic.dll");
            assemblies.Add("Microsoft.VisualBasic.Vsa.dll");
            assemblies.Add("Microsoft.Vsa.dll");
            assemblies.Add("Microsoft.Vsa.Vb.CodeDOMProcessor.dll");
            assemblies.Add("Microsoft_VsaVb.dll");
            assemblies.Add("mscorlib.dll");
            assemblies.Add("sysglobl.dll");
            assemblies.Add("System.configuration.dll");
            assemblies.Add("System.Configuration.Install.dll");
            assemblies.Add("System.Data.dll");
            assemblies.Add("System.Data.OracleClient.dll");
            assemblies.Add("System.Data.SqlXml.dll");
            assemblies.Add("System.Deployment.dll");
            assemblies.Add("System.Design.dll");
            assemblies.Add("System.DirectoryServices.dll");
            assemblies.Add("System.DirectoryServices.Protocols.dll");
            assemblies.Add("System.dll");
            assemblies.Add("System.Drawing.Design.dll");
            assemblies.Add("System.Drawing.dll");
            assemblies.Add("System.EnterpriseServices.dll");
            assemblies.Add("System.Management.dll");
            assemblies.Add("System.Messaging.dll");
            assemblies.Add("System.Runtime.Remoting.dll");
            assemblies.Add("System.Runtime.Serialization.Formatters.Soap.dll");
            assemblies.Add("System.Security.dll");
            assemblies.Add("System.ServiceProcess.dll");
            assemblies.Add("System.Transactions.dll");
            assemblies.Add("System.Web.dll");
            assemblies.Add("System.Web.Mobile.dll");
            assemblies.Add("System.Web.RegularExpressions.dll");
            assemblies.Add("System.Web.Services.dll");
            assemblies.Add("System.Windows.Forms.dll");
            assemblies.Add("System.XML.dll");
            assemblies.Add("vjscor.dll");
            assemblies.Add("vjsjbc.dll");
            assemblies.Add("vjslib.dll");
            assemblies.Add("vjslibcw.dll");
            assemblies.Add("vjssupuilib.dll");
            assemblies.Add("vjsvwaux.dll");
            assemblies.Add("vjswfc.dll");
            assemblies.Add("VJSWfcBrowserStubLib.dll");
            assemblies.Add("vjswfccw.dll");
            assemblies.Add("vjswfchtml.dll");


            return;
        }
    }
}

That would be a great help Tarks, thanks! Continuing along this topic, is it possible to turn a c# block of code into a string? — NoizWaves, Dec 06 '08 at 09:25

Daniel Earwicker · Answer 5 · 2008-12-06T10:04:51.513

1

Compile it into a separate assembly, send the assembly, have the other process load it.

You might want to consider security implications.

Update: another idea would be to generate an expression tree and use this library to serialize it:

http://www.codeplex.com/metalinq/

edited Dec 06 '08 at 10:04

answered Dec 06 '08 at 09:19

Daniel Earwicker

114,894
38
205
284

score 1 · Answer 6 · answered Dec 06 '08 at 13:37

It is an interesting challenge, but you should probably describe why you want to do this, since there is a lot of different approaches depending on your objective. As humpohl points out, there is also some pretty serious security issues.

"Serialized code" could just be source code or a compiled assembly, depending on your requirements. You probably don't need to use a seperate code serialization format.

If you want to generate code dynamically and pass that on, you could generate code using CodeDOM and compile it. However, you most likely dont need to generate completely arbitrary code.

score 0 · Answer 7 · edited Apr 15 '12 at 21:44

0

Another option is using the DLR, and constraining the code to execute...

edited Apr 15 '12 at 21:44

Peter Mortensen

30,738
21
105
131

answered Dec 06 '08 at 09:26

Tracker1

19,103
12
80
106

Is it possible to serialize a C# code block?

7 Answers7

Linked