How to pre-load all deployed assemblies for an AppDomain

Question

UPDATE: I now have a solution I'm much happier with that, whilst not solving all the problems I ask about, it does leave the way clear to do so. I've updated my own answer to reflect this.

Original Question

Given an App Domain, there are many different locations that Fusion (the .Net assembly loader) will probe for a given assembly. Obviously, we take this functionality for granted and, since the probing appears to be embedded within the .Net runtime (Assembly._nLoad internal method seems to be the entry-point when Reflect-Loading - and I assume that implicit loading is probably covered by the same underlying algorithm), as developers we don't seem to be able to gain access to those search paths.

My problem is that I have a component that does a lot of dynamic type resolution, and which needs to be able to ensure that all user-deployed assemblies for a given AppDomain are pre-loaded before it starts its work. Yes, it slows down startup - but the benefits we get from this component totally outweight this.

The basic loading algorithm I've already written is as follows. It deep-scans a set of folders for any .dll (.exes are being excluded at the moment), and uses Assembly.LoadFrom to load the dll if it's AssemblyName cannot be found in the set of assemblies already loaded into the AppDomain (this is implemented inefficiently, but it can be optimized later):

void PreLoad(IEnumerable<string> paths)
{
  foreach(path p in paths)
  {
    PreLoad(p);
  }
}

void PreLoad(string p)
{
  //all try/catch blocks are elided for brevity
  string[] files = null;

  files = Directory.GetFiles(p, "*.dll", SearchOption.AllDirectories);

  AssemblyName a = null;
  foreach (var s in files)
  {
    a = AssemblyName.GetAssemblyName(s);
    if (!AppDomain.CurrentDomain.GetAssemblies().Any(
        assembly => AssemblyName.ReferenceMatchesDefinition(
        assembly.GetName(), a)))
      Assembly.LoadFrom(s);
  }    
}

LoadFrom is used because I've found that using Load() can lead to duplicate assemblies being loaded by Fusion if, when it probes for it, it doesn't find one loaded from where it expects to find it.

So, with this in place, all I now have to do is to get a list in precedence order (highest to lowest) of the search paths that Fusion is going to be using when it searches for an assembly. Then I can simply iterate through them.

The GAC is irrelevant for this, and I'm not interested in any environment-driven fixed paths that Fusion might use - only those paths that can be gleaned from the AppDomain which contain assemblies expressly deployed for the app.

My first iteration of this simply used AppDomain.BaseDirectory. This works for services, form apps and console apps.

It doesn't work for an Asp.Net website, however, since there are at least two main locations - the AppDomain.DynamicDirectory (where Asp.Net places it's dynamically generated page classes and any assemblies that the Aspx page code references), and then the site's Bin folder - which can be discovered from the AppDomain.SetupInformation.PrivateBinPath property.

So I now have working code for the most basic types of apps now (Sql Server-hosted AppDomains are another story since the filesystem is virtualised) - but I came across an interesting issue a couple of days ago where this code simply doesn't work: the nUnit test runner.

This uses both Shadow Copying (so my algorithm would need to be discovering and loading them from the shadow-copy drop folder, not from the bin folder) and it sets up the PrivateBinPath as being relative to the base directory.

And of course there are loads of other hosting scenarios that I probably haven't considered; but which must be valid because otherwise Fusion would choke on loading the assemblies.

I want to stop feeling around and introducing hack upon hack to accommodate these new scenarios as they crop up - what I want is, given an AppDomain and its setup information, the ability to produce this list of Folders that I should scan in order to pick up all the DLLs that are going to be loaded; regardless of how the AppDomain is setup. If Fusion can see them as all the same, then so should my code.

Of course, I might have to alter the algorithm if .Net changes its internals - that's just a cross I'll have to bear. Equally, I'm happy to consider SQL Server and any other similar environments as edge-cases that remain unsupported for now.

Any ideas!?

Answer 1

I have now been able to get something much closer to a final solution, except it's still not processing the private bin path correctly. I have replaced my previously live code with this and have also solved a few nasty runtime bugs I've had into the bargain (dynamic compilation of C# code referencing far too many dlls).

The golden rule I've since discovered is always use the load context, not the LoadFrom context, since the Load context will always be the first place .Net looks when performing a natural bind. Therefore, if you use the LoadFrom context, you will only get a hit if you actually load it from the same place that it would naturally bind it from - which isn't always easy.

This solution works both for web applications, taking into account the bin folder difference versus 'standard' apps. It can easily be extended to accommodate the PrivateBinPath problem, once I can get a reliable handle on exactly how it is read(!)

private static IEnumerable<string> GetBinFolders()
{
  //TODO: The AppDomain.CurrentDomain.BaseDirectory usage is not correct in 
  //some cases. Need to consider PrivateBinPath too
  List<string> toReturn = new List<string>();
  //slightly dirty - needs reference to System.Web.  Could always do it really
  //nasty instead and bind the property by reflection!
  if (HttpContext.Current != null)
  {
    toReturn.Add(HttpRuntime.BinDirectory);
  }
  else
  {
    //TODO: as before, this is where the PBP would be handled.
    toReturn.Add(AppDomain.CurrentDomain.BaseDirectory);
  }

  return toReturn;
}

private static void PreLoadDeployedAssemblies()
{
  foreach(var path in GetBinFolders())
  {
    PreLoadAssembliesFromPath(path);
  }
}

private static void PreLoadAssembliesFromPath(string p)
{
  //S.O. NOTE: ELIDED - ALL EXCEPTION HANDLING FOR BREVITY

  //get all .dll files from the specified path and load the lot
  FileInfo[] files = null;
  //you might not want recursion - handy for localised assemblies 
  //though especially.
  files = new DirectoryInfo(p).GetFiles("*.dll", 
      SearchOption.AllDirectories);

  AssemblyName a = null;
  string s = null;
  foreach (var fi in files)
  {
    s = fi.FullName;
    //now get the name of the assembly you've found, without loading it
    //though (assuming .Net 2+ of course).
    a = AssemblyName.GetAssemblyName(s);
    //sanity check - make sure we don't already have an assembly loaded
    //that, if this assembly name was passed to the loaded, would actually
    //be resolved as that assembly.  Might be unnecessary - but makes me
    //happy :)
    if (!AppDomain.CurrentDomain.GetAssemblies().Any(assembly => 
      AssemblyName.ReferenceMatchesDefinition(a, assembly.GetName())))
    {
      //crucial - USE THE ASSEMBLY NAME.
      //in a web app, this assembly will automatically be bound from the 
      //Asp.Net Temporary folder from where the site actually runs.
      Assembly.Load(a);
    }
  }
}

First we have the method used to retrieve our chosen 'app folders'. These are the places where the user-deployed assemblies will have been deployed. It's an IEnumerable because of the PrivateBinPath edge case (it can be a series of locations), but in practise it's only ever one folder at the moment:

The next method is PreLoadDeployedAssemblies(), which gets called before doing anything (here it's listed as private static - in my code this is taken from a much larger static class that has public endpoints that will always trigger this code to be run before doing anything for the first time.

Finally there's the meat and bones. The most important thing here is to take an assembly file and get it's assembly name, which you then pass to Assembly.Load(AssemblyName) - and not to use LoadFrom.

I previously thought that LoadFrom was more reliable, and that you had to manually go and find the temporary Asp.Net folder in web apps. You don't. All you have to is know the name of an assembly that you know should definitely be loaded - and pass it to Assembly.Load. After all, that's practically what .Net's reference loading routines do :)

Equally, this approach works nicely with custom assembly probing implemented by hanging off the AppDomain.AssemblyResolve event as well: Extend the app's bin folders to any plugin container folders you may have so that they get scanned. Chances are you've already handled the AssemblyResolve event anyway to ensure they get loaded when the normal probing fails, so everything works as before.

Answer 2

This is what I do:

public void PreLoad()
{
    this.AssembliesFromApplicationBaseDirectory();
}

void AssembliesFromApplicationBaseDirectory()
{
    string baseDirectory = AppDomain.CurrentDomain.BaseDirectory;
    this.AssembliesFromPath(baseDirectory);

    string privateBinPath = AppDomain.CurrentDomain.SetupInformation.PrivateBinPath;
    if (Directory.Exists(privateBinPath))
        this.AssembliesFromPath(privateBinPath);
}

void AssembliesFromPath(string path)
{
    var assemblyFiles = Directory.GetFiles(path)
        .Where(file => Path.GetExtension(file).Equals(".dll", StringComparison.OrdinalIgnoreCase));

    foreach (var assemblyFile in assemblyFiles)
    {
        // TODO: check it isnt already loaded in the app domain
        Assembly.LoadFrom(assemblyFile);
    }
}

Answer 3

Have you tried looking at Assembly.GetExecutingAssembly().Location? That should give you the path to the assembly where your code is running from. In the NUnit case, I would expect that to be where the assemblies were shadow copied to.

Answer 4

Based on the answer I went and expanded the solution to a reusable utility with programmable functions to change default assembly discovery behavior. Please correct me if subfolders should be included in the discovery.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Text;

namespace TheOperator.Foundation.Features
{
    public static class AssemblyLoading
    {
        public static bool IsAssemblyFile(
            string path)
        {
            return File.Exists(path) && Path.GetExtension(path).Equals(".dll", StringComparison.OrdinalIgnoreCase);
        }

        public static Assembly[] TryLoadFromDirectory(
            string path)
            => TryLoadFromDirectoryCore(path).ToArray();

        private static IEnumerable<Assembly> TryLoadFromDirectoryCore(
            string path)
        {
            if (Directory.Exists(path))
            {
                foreach (var assemblyFile in DiscoverFromDirectory(path))
                {
                    if (!IsLoaded(assemblyFile))
                    {
                        yield return Assembly.LoadFrom(assemblyFile);
                    }
                }
            }
        }

        public static Func<IEnumerable<Assembly>> GetLoaded
            = () => AppDomain.CurrentDomain.GetAssemblies();

        public static Func<string, IEnumerable<string>> DiscoverFromDirectory
            = x => Directory.GetFiles(x).Where(xx => IsAssemblyFile(xx));

        public static Action<List<string>> CollectDiscoveryDirectories
            = x =>
            {
                x.Add(AppDomain.CurrentDomain.BaseDirectory);
                x.Add(AppDomain.CurrentDomain.SetupInformation.PrivateBinPath);
            };

        public static void TryLoadFromDiscoveryDirectories()
        {
            var locations = new List<string>();
            CollectDiscoveryDirectories(locations);
            foreach (var location in locations)
            {
                TryLoadFromDirectory(location);
            }
        }

        public static bool TryLoadFromFile(
            string path)
        {
            if (IsAssemblyFile(path))
            {
                if (!IsLoaded(path))
                {
                    Assembly.Load(path);
                    return true;
                }
                else
                {
                    return false;
                }
            }
            else
            {
                return false;
            }
        }

        public static bool IsLoaded(
            string assembly)
        {
            foreach (var loadedAssembly in GetLoaded())
            {
                if (!loadedAssembly.IsDynamic && loadedAssembly.Location.Equals(assembly, StringComparison.OrdinalIgnoreCase))
                {
                    return true;
                }
            }
            return false;
        }
    }
}