Run application in context of multiple directories in order to use a base set of assemblies and an external set of assemblies

Question

The problem and current solution

A business app needs to interface with a 3rd party app which provides external client data. The 3rd party API is through client-specific assemblies which are provided by the clients and stored locally. Across all clients the assemblies are known and have the same interface, but each client may have customizations that result in different data, so it's necessary to run the app in the context of each set of assemblies.

The current solution is to house each client's set of assemblies in their own folder with an IIS web app on top of it. The business app call a particular client IIS web app through a local web request, which loads in the client's assemblies: This solution works, but is a lot of overhead to maintain on the server, and adds complexity to the application and deployment. There are hundreds of clients, and 1 IIS web app per client.

The Goal

I would like to get rid of the IIS web apps and just have the business app load a particular client's assemblies at runtime from their directory. Something like: I have been looking into AppDomains, but it feels like that is not quite the right solution for this. There are 5-ish different request types that happen, and within each request, several API calls that are made to the client application. The API calls are a mix of instance and static methods, which has proven challenging to do with AppDomains.

Again, I don't know if this is possible, but it seems like I'm looking for something like:

OnAssemblyLoad(assemblyName =>
{
  if(assemblyName.StartsWith("ClientAssembly"))
  {
    return "clients\clientA";
  }
  else
  {
    return "[executing directory]";
  }
});

Or some way I can create a mapping for a set of assemblies. As an extra wrench, a set of the client assemblies are in the executing directory (for compile reference), so I'll need to ignore those and use the specific version from the other directory.

I hope I've explained the problem and desired solution well, but if anything is unclear please let me know, and thanks!

Answer 1

It took many iterations of attempts, but I finally found something that works. I had to abandon AppDomain.AssemblyResolve, but the solution is logically basically what I was hoping for.

The generic method (and relevant field & Dispose) I wrote to get client data depending on the request:

private AppDomain ClientDomain = null;

public void Dispose()
{
    if(this.ClientDomain != null)
    {
        AppDomain.Unload(this.ClientDomain);
    }
}

private T CallClientAppDomain<S, T>(Type requestType, Func<S, T> requestFunc)
{
    if (this.ClientDomain == null)
    {
        AppDomainSetup setup = new AppDomainSetup
        {
            ApplicationBase = "parent\directory\that\can\reach\business\app\and\client\directory",
            PrivateBinPath = $"relative\to\parent\business-app\bin;relative\to\parent\client-directory\{ClientId}\bin"
        };

        this.ClientDomain = AppDomain.CreateDomain(
            $"ClientDomain{ClientId}",
            AppDomain.CurrentDomain.Evidence,
            setup
        );
    }

    S request = (S)this.ClientDomain.CreateInstanceAndUnwrap(
        requestType.Assembly.FullName,
        requestType.FullName,
        false,
        BindingFlags.Default,
        null,
        new object[] { constructorParam1, constructorParam2 },
        System.Globalization.CultureInfo.InvariantCulture,
        new object[0]
    );

    return requestFunc(request);
}

And a caller looks like:

var clientData = this.CallClientAppDomain(
    typeof(SeparateAssembly.ServiceClass),
    // IServiceClass defined in SharedAssembly. SeparateAssembly.ServiceClass implements IServiceClass
    (IServiceClass request) =>
    {
        return request.GetData();
    }
);

Some things I discovered along the way:

• As Sean mentioned above, the ClientAssemblies can be referenced in code, but need to be marked "Copy Local = False" to get them out of the executing directory and manually load them for each client.
• AppDomain.CurrentDomain is not scoped or instanced to the class instance in any way, so once a client's assemblies were loaded into it, they were stuck there and would be used for another client
• AppDomain.AssemblyResolve worked fine in a console app, but absolutely refused to work in a .NET Framework web app (either WCF or Web API). Even attempting to set it would throw a serialization error like "type not loaded for member".
• A console app could handle ServiceClass and the caller being defined in the same assembly, but the key to get it working through a web app was to move ServiceClass to SeparateAssembly. For code nicety, I also created SharedAssembly that has interfaces/models that SeparateAssembly.ServiceClass implements/returns
• SeparateAssembly.ServiceClass, along with the type that it returns, has to inherit from MarshalByRefObject (or be in a hierarchy that does) in order to actually execute within the separate AppDomain and return data to the home domain. Otherwise, it attempts to execute in AppDomain.CurrentDomain
• The constructor params passed to ServiceClass need to be marked [Serializable], as well as classes they use as properties, including any base classes they inherit from all the way back up the chain (only marking the derived class as Serializable does not work). The serialization error that gets thrown is not very useful in telling you which part of the param is not correctly marked Serializable, but I found this method to manually serialize the object more useful: https://stackoverflow.com/a/236698

Thanks to Sean for pointing me in the right direction that eventually led to a solution!