How can I "start" a DLL like an executable at runtime?

Question

I want to write a very, very small program that parses the launch arguments and chooses one of several DLLs to "boot into."

I've already written an application that I'd like to "run" as a DLL by writing it as an application, then changing the Visual Studio project properties to build it as a DLL instead. I know I need to use LoadLibrary and GetProcAddress in concert to get the functionality I want, but I'm having trouble finding clear and comprehensive documentation on this, as a lot of the use cases aren't really of this nature. Also, I have to go this route based on the project and platform restrictions.

I found this page, which has some information, but it's not clear enough for me to adapt for my purposes.

Edit: Here's where I'm at right now.

I have a DLL project whose main function signature looks something like this:

__declspec(dllexport) int cdecl main(int argc, char *argv[])

I also have an application project whose attempt at loading the DLL and running the above function looks like this:

typedef int (CALLBACK* LPFNDLLFUNC1)(int, char *);

...

        HMODULE dllHandle = NULL;
        BOOL freeResult, runTimeLinkSuccess = FALSE;
        LPFNDLLFUNC1 lpfnDllFunc1;    // Function pointer  
        if (args->IsEmpty())
        {
            dllHandle = LoadLibrary(L"TrueApplication.dll");
            if (NULL != dllHandle)
            {
                lpfnDllFunc1 = (LPFNDLLFUNC1)GetProcAddress(dllHandle, "main");
                if (lpfnDllFunc1)
                {
                    int retVal = lpfnDllFunc1(0, "1");
                }

Currently, the LoadLibrary call works, but not GetProcAddress.

Answer 1

You don't have to use LoadLibrary and GetProcAddress to invoke the functionality in the DLL.

More often, you'd create your DLLs, each with its own entry point. For the moment, let's assume you want to parse the command line, choose a DLL, and invoke its entry point without arguments. You end up with something like this:

void DLL_a();
void DLL_b();
void DLL_c();

int main(int argc, char **argv) { 
    // we'll assume DLL_a is the default:
    if (argc < 2) 
        DLL_a();

    // For now, we'll do a *really* trivial version of parsing the command line
    // to choose the right DLL:
    if (argv[1][0] == 'A')
        DLL_a();
    else if (argv[1]][0] == 'B')
        DLL_b();
    else if (argv[1][0] == 'C')
        DLL_c();
    else {
        std::cerr << "Unrecognized argument\n";
        return 1;
    }
}

When you link your main, you'll specify the .lib corresponding to each DLL, and you'll probably want to specify the /delayload flag to the linker. This means a DLL won't be loaded until a function in the DLL is actually invoked. If, for example, you want to distribute a reduced-functionality version of your program that only includes DLL A, it'll still be able run (without DLL B or C present on the user's system) as long as no function from DLL B or C is ever called. If you don't specify /delayload, the loader will attempt to map all the DLLs to RAM as the program starts up, run their DllMain to initialize them for use, do the same recursively for all the DLLs they depend on, etc.

/delayload has one other advantage: it avoids mapping the other DLLs to addresses at all if they're never used. It sounds like any given invocation will only use one DLL, so that's probably a win in your case.

Answer 2

First of all, changing project type from executable to DLL is not enough to make a DLL. You also need to export some symbols to create your API. At least, you need to decorate the functions you are exporting with __declspec(dllexport). However, I recommend that you export C API, meaning extern "C" functions with C-compatible arguments. So, the functions you export should be prepended with extern "C" __declspec(dllexport).

Once you have done that, you can dynamically load your DLL like this:

   const char* dllname = "myfile.dll";
   h = LoadLibrary(dllname);
   if( h == nullptr )
   {
       /*handle error*/
   }

   using myfunc_type = bool (*)(int x, double y); //example
   auto myfunc = reinterpret_cast<myfunc_type>(GetProcAddress(h, "myfunc"));       
   //......
   myfunc(x,y); //call the imported function

This solution takes more work than static loading with /delayload shown by Jerry Coffin, but it has an advantage: if a DLL is required but not found, you can give users your own error message instead of relying on the message coming from Windows (which is often unacceptable for non-tech people). You can also include API version verification with its own custom error message in the API.

Edit: the code sample will work if you change it like this

extern "C" __declspec(dllexport) int main(int argc, char *argv[]){...}
typedef int (* LPFNDLLFUNC1)(int, char **);

Answer 3

You do not need GetProcAddress (...) to do this, though that approach (Option #2) is simpler once you understand how the compiler generates symbol names.

---

Option #1

DllMain Spawns a Main Thread

Never do anything complicated inside of DllMain, you may deadlock your software.

DLLs have their own entry-point (and exit-point and thread attach-point ... it's a really busy function). Just calling LoadLibrary (...) on your DLL causes at minimum one call to DllMain (...) for process attach.

BOOL
APIENTRY
DllMain ( HMODULE hModule,
          DWORD   ul_reason_for_call,
          LPVOID  lpReserved )

You can actually treat ul_reason_for_call == DLL_PROCESS_ATTACH as a mandate to execute DllMain as if it were your program's main function.

Now, by no means should you actually start a program loop here... anytime DllMain runs it holds a very important operating system lock (DLL Loader) and you need to release that by returning for normal program operation.

That means if you want to use DllMain as your program's entry-point, it needs to spawn a thread and your original main method must not return until that thread finishes...

---

Option #2

DLL Exports a main Function.

Be very mindful of calling convention, the compiler will rename the symbols for you and make locating functions in a DLL with GetProcAddress less than intuitive.

In your DLL, export main:

__declspec (dllexport)
int
__cdecl main (int argc, char *argv [])
{
  printf ("foobar");
  return 0;
}

In your program, import main from the DLL:

// Need a typedef for the function you are going to get from the DLL
typedef int (__cdecl *main_pfn)(int argc, char *argv[]);

int main (int argc, char *argv[])
{
  HMODULE hModMyDLL = LoadLibraryA ("MyDll.dll");

  if (hModMyDLL != 0) {
    //
    // The preceding underscore deals with automatic decorations
    //   the compiler added to the __cdecl function name.
    //
    //  It is possible to do away with this completely if you use a .def
    //    file to assign export names and ordinals manually, but then you
    //      lose the ability to tell a function's calling convention by its
    //        name alone.
    //
    main_pfn MyMain = (main_pfn)
      GetProcAddress (hModMyDLL, "_main");

    // Call the main function in your DLL and return when it does
    if (MyMain != nullptr)
      return MyMain (argc, argv);
  }

  return -1;
}

---

Both approaches have their merits.

Spawning a thread from DllMain avoids knowing anything at all about how the DLL you want to load is is implemented, but it also requires you to design your main function never to return -- the DLL will call ExitProcess (...).

Exporting functions and later importing them by name allows you to avoid pussyfooting around the Windows DLL loaderlock. However, if you do not use a .def file to explicitly name the exported symbols, the compiler is going to add decorations such as _... (__cdecl) or ...@n (__stdcall) to the names and you have to learn these conventions to do anything useful with GetProcAddress.