Unable to call method in managed dll after loading CLR. Why?

Question

I have met some issues when trying to inject a dll into a process. I am quite new at the topic but am familiar with C# so reading and understand the syntax of C++ wasnt that unfamiliar and i understand it for the most part.

What i am trying is only for learning, and i am trying this with simple applications like notepad.exe and calc.exe.

Project setup:

WPF application - To pick the process i want to tinker with and inject the unmanaged dll.
CppDLL.dll - Unmanaged dll to load CLR, managed dll and call method on managed dll.
SharpDLL.dll - Managed dll.

(wpf) c# of interest

dllToInject = fileDialog.FileName;
Process targetProcess = Process.GetProcessById(processToInject.ID);
var dllInjector = DllInjector.GetInstance;
DllInjectionResult injectResult;
if ((injectResult = dllInjector.Inject(processToInject.Name,dllToInject)) == DllInjectionResult.Success)
{
    MessageBox.Show("Success");
} else
{
    MessageBox.Show("Error: " + injectResult.ToString());
}

The unmanaged dll is successfully injected when not trying to load clr and managed dll as shown below.

But when i try to load CLR and managed dll it fails.

CppDLL.dll dllmain.cpp :

#include "stdafx.h"
#include <Windows.h>
#include <metahost.h>
#pragma comment(lib, "mscoree.lib")

#import "mscorlib.tlb" raw_interfaces_only \
    high_property_prefixes("_get","_put","_putref") \
    rename("ReportEvent", "InteropServices_ReportEvent")

void LoadDotNet()
{
  HRESULT hr;
  ICLRMetaHost *pMetaHost = NULL;
  ICLRRuntimeInfo *pRuntimeInfo = NULL;
  ICLRRuntimeHost *pClrRuntimeHost = NULL;

  hr = CLRCreateInstance(CLSID_CLRMetaHost, IID_PPV_ARGS(&pMetaHost));
  hr = pMetaHost->GetRuntime(L"v4.0.30319", IID_PPV_ARGS(&pRuntimeInfo));
  hr = pRuntimeInfo->GetInterface(CLSID_CLRRuntimeHost,
    IID_PPV_ARGS(&pClrRuntimeHost));
  hr = pClrRuntimeHost->Start();

  DWORD pReturnValue;
  hr = pClrRuntimeHost->ExecuteInDefaultAppDomain(
    L"C:\\Users\\DanHovedPC\\Desktop\\inject\\SharpDLL.dll",
    L"SharpDLL.Injected",
    L"Start",
    L"Hello from .NET",
    &pReturnValue);

  pMetaHost->Release();
  pRuntimeInfo->Release();
  pClrRuntimeHost->Release();
}

BOOL APIENTRY DllMain( HMODULE hModule,
                       DWORD  ul_reason_for_call,
                       LPVOID lpReserved
           )
{
  switch (ul_reason_for_call)
  {
  case DLL_PROCESS_ATTACH:
    MessageBox(NULL, L"Hi!", L"From cpp DLL", NULL);
    //LoadDotNet();
  case DLL_THREAD_ATTACH:
  case DLL_THREAD_DETACH:
  case DLL_PROCESS_DETACH:
    break;
  }
  return TRUE;
}

SharpDLL.dll Injected.cs

using System.Windows;
namespace SharpDLL
{
    class Injected
    {
        public static int Start(string arg)
        {
            MessageBox.Show(arg);
            return 0;
        }
    }
}

In CppDLL.dll if i uncomment the function and comment the messagebox it fails. The SharpDLL.dll does not get injected. And when i try to close notepad the process still shows up in Process Explorer.

I have looked at the process in Process Explorer beforehand and the clr.dll is not loaded by default, but it gets loaded when the function runs. Maybe it could be the .NET version? I am running Windows 10 x64.

Update

The code runs until i try to actually start the runtime

void LoadDotNet()
{
  HRESULT hr;
  ICLRMetaHost *pMetaHost = NULL;
  ICLRRuntimeInfo *pRuntimeInfo = NULL;
  ICLRRuntimeHost *pClrRuntimeHost = NULL;

  hr = CLRCreateInstance(CLSID_CLRMetaHost, IID_PPV_ARGS(&pMetaHost));
  hr = pMetaHost->GetRuntime(L"v4.0.30319", IID_PPV_ARGS(&pRuntimeInfo));
  hr = pRuntimeInfo->GetInterface(CLSID_CLRRuntimeHost,
    IID_PPV_ARGS(&pClrRuntimeHost));

  // start runtime
  MessageBox(NULL, L"Runs up to here...", L"DEBUG", NULL);
  hr = pClrRuntimeHost->Start();
  MessageBox(NULL,(LPCWSTR)GetLastError(),L"DEBUG",NULL);
  pMetaHost->Release();
  pRuntimeInfo->Release();
  pClrRuntimeHost->Release();
}

The first messagebox shows.

score 0 · Answer 1 · edited May 23 '17 at 12:23

I figured out that the problem was that code within DllMain must not access the CLR.

Code within DllMain must not access the CLR. This means that DllMain should make no calls to managed functions, directly or indirectly; no managed code should be declared or implemented in DllMain; and no garbage collection or automatic library loading should take place within DllMain.

https://msdn.microsoft.com/en-us/library/ms173266.aspx

When creating a new thread the code runs successfully

BOOL APIENTRY DllMain(HMODULE hModule,
  DWORD  ul_reason_for_call,
  LPVOID lpReserved
  )
{
  switch (ul_reason_for_call)
  {
  case DLL_PROCESS_ATTACH:
    //printf("DLL Loaded!");
    CreateThread(NULL, 0,
      (LPTHREAD_START_ROUTINE)LoadDotNet, NULL, 0, NULL);
  }
  return TRUE;
}

And from reading a comment on another question here on SO CreateThread should be safe in this particular case.

Unable to call method in managed dll after loading CLR. Why?

1 Answers1