Executing rundll32.exe with CreateProcess

Question

I've created a DLL and would like to execute one of the functions using the rundll32.exe command on windows.

Using rundll32.exe, it runs correctly from the command line; however, I'd like to call it (rundll32.exe) from a separate program. I cannot directly call the function from my code due to 32/64 bit compatibility issues in the underlying libraries I'm using (Easyhook).

Below is what I'm using in an attempt to run the dll function:

STARTUPINFO si;
PROCESS_INFORMATION pi;

ZeroMemory( &si, sizeof(si));
si.cb = sizeof(si);
ZeroMemory( &pi, sizeof(pi));

LPCTSTR application = "C:\\Windows\\system32\\rundll32.exe";
LPTSTR cmd = "C:\\Projects\\Test\\mydll.dll,MyFunc";

BOOL cpRes = CreateProcess(application,
  cmd,
  NULL,
  NULL,
  FALSE,
  0,
  NULL,
  NULL,
  &si,
  &pi);

if(cpRes == 0) {
  cout << "ERROR\n";
  cout << GetLastError() << endl;
} else {
  cout << "DLL Launched!" << endl;
}

CloseHandle(pi.hProcess);
CloseHandle(pi.hThread);

The output to my console is always DLL Launched; however, I do not see the effects of my DLL actually being called (currently stubbed out in such a way that the command writes to a file).

If I swap out the application with something such as C:\\Windows\\system32\\notepad.exe, the program successfully runs.

For completion, here's the body of MyFunc:

ofstream file;
file.open("C:\\Projects\\Test\\test.txt");
file << "I wrote to a file!";
file.close();

Is there any reason CreateProcess cannot be used with rundll32? While reading over this I found several warnings about LoadLibrary() and DLLMain but it doesn't seem like they're relevant to this.

---

More Clarification:
This is currently a 32-bit application (allegedly) launching the 32-bit rundll32.exe (Logic will be added later to call the 32 or 64 bit version).

My dll is as follows:

extern "C" __declspec(dllexport) void CALLBACK MyFunc(HWND hwnd, HINSTANCE hinst, LPSTR lpszCmdLine, int nCmdShow);

void CALLBACK MyFunc(HWND hwnd, HINSTANCE hinst, LPSTR lpszCmdLine, int nCmdShow) { ... }

Which also has a .def file with:

EXPORTS
  MyFunc

Running

C:\Windows\system32\rundll32.exe C:\Projects\Test\mydll.dll,MyFunc 

produces the expected results.

---

Update
Setting application to NULL and including the rundll32.exe in cmd as mentioned in the comments seems to work.

Relevant Docs:
CreateProcess
RunDll32.exe

Answer 1

Per the CreateProcess() documentation:

If both lpApplicationName and lpCommandLine are non-NULL, the null-terminated string pointed to by lpApplicationName specifies the module to execute, and the null-terminated string pointed to by lpCommandLine specifies the command line. The new process can use GetCommandLine to retrieve the entire command line. Console processes written in C can use the argc and argv arguments to parse the command line. Because argv[0] is the module name, C programmers generally repeat the module name as the first token in the command line.

You are not repeating rundll32.exe as the first command-line token.

So, if you continue using the lpApplicationName parameter, then change this:

LPCTSTR application = "C:\\Windows\\system32\\rundll32.exe";
LPTSTR cmd = "C:\\Projects\\Test\\mydll.dll,MyFunc";

To this instead:

LPCTSTR application = TEXT("C:\\Windows\\system32\\rundll32.exe");
LPTSTR cmd = TEXT("C:\\Windows\\system32\\rundll32.exe C:\\Projects\\Test\\mydll.dll,MyFunc");

Note that you are currently compiling for ANSI/MBCS (by virtue of the fact that you are passing narrow strings to CreateProcess()). If you ever update the project to compile for Unicode, use this instead:

TCHAR cmd[] = TEXT("C:\\Windows\\system32\\rundll32.exe C:\\Projects\\Test\\mydll.dll,MyFunc");

This is because the documentation states:

lpCommandLine [in, out, optional]
...
The Unicode version of this function, CreateProcessW, can modify the contents of this string. Therefore, this parameter cannot be a pointer to read-only memory (such as a const variable or a literal string). If this parameter is a constant string, the function may cause an access violation.

You might consider changing cmd into a TCHAR[] array anyway, even in ANSI/MBCS, so you can do something like this:

LPCTSTR application = TEXT("C:\\Windows\\system32\\rundll32.exe");

TCHAR cmd[(MAX_PATH*2)+10];
wsprintf(cmd, TEXT("%s %s,%s"), application, TEXT("C:\\Projects\\Test\\mydll.dll"), TEXT("MyFunc"));

Either way, by passing the module filename as the first token in the lpCommandLine parameter, you can then set the lpApplicationName parameter to NULL:

The lpApplicationName parameter can be NULL. In that case, the module name must be the first white space–delimited token in the lpCommandLine string.

Let CreateProcess() setup the correct command-line to pass to rundll32.exe for you:

TCHAR cmd[] = TEXT("C:\\Windows\\system32\\rundll32.exe C:\\Projects\\Test\\mydll.dll,MyFunc");

BOOL cpRes = CreateProcess(NULL, cmd, ...);