Debugging a native dll using rundll32.exe , fail to load symbols

Question

I'm writing a native dll that is due to run with rundll32.exe (that is obligated by our clients). I've using VS's debugging properties to define:

Command: c:\windows\system32\rundll32.exe
Command Argument: $(TargetPath) , ENTRY_POINT

where ENTRY_POINT is an exported function of my dll, that obey the rundll32.exe interface.

This setup calls my function , but won't load any symbols and thus won't trigger any break point. I've learned that my function was called only after placing a call to MessageBox at its entrance.

when I use my own container application (just an exe calling Loadlibrary , GetProcAddress and the ENTRY_POINT function itself) all break points are triggered, and a step-by-step debugging is possible as usual.

what can cause that behavior?

You are debugging the process *rundll32.exe*, and Visual Studio does not know that this process will eventually load a .dll that is the result of compiling the current solution. You cannot set breakpoints using the source code editor. The alternative is to use [`DebugBreak`](http://msdn.microsoft.com/en-us/library/windows/desktop/ms679297.aspx), if a debugger is attached ([`IsDebuggerPresent`](http://msdn.microsoft.com/en-us/library/windows/desktop/ms680345.aspx)). — IInspectable, Dec 26 '13 at 14:55
In old visual studio (6.0) there was an option to specify additional DLLs that can be debugged. This is not seen in VS2010, 12 etc. Have you got the DLL and PDB in the same folder? That should work normally. Another workaround would be that you could put a sleep after the DLL's entry function and attach the debugger to the rundll process quickly during this time. — PermanentGuest, Dec 26 '13 at 15:08
@IInspectable But OP is setting debug properties of a DLL project by running an EXE which will load the DLL that he just compiled in debug mode. Break points should fire... — manuell, Dec 26 '13 at 15:11
@IInspectable Your idea showed me that the debugger was not attached automatically, which now also makes sense to me. I used PermanentGuest idea to sleep and manully attach. so now `DebugBreak()` triggers the `VS` debugger, but still no symbols loaded , and the first `F11` click sends me to assembly. Also , I've checked and `.pdb`'s are on the same directory as the binaries. — friedrich kuhler, Dec 26 '13 at 15:17
@friedrich Did you run *rundll32.dll* under a debugger? If so, `IsDebuggerPresent` should return `TRUE` in your .dll. Using your strategy, you should be able to right-click a method in the callstack and load symbols from there. For diagnostics make sure you have enabled source server support and have it print diagnostic messages (Tools -> Options -> Debugging -> General). — IInspectable, Dec 26 '13 at 15:22
You probably should [abandon rundll32 altogether](http://blogs.msdn.com/b/oldnewthing/archive/2013/01/04/10382242.aspx). — Christian.K, Dec 27 '13 at 08:46

score 2 · Answer 1 · answered Dec 26 '13 at 16:04

The MessageBox() gives you enough rope to troubleshoot this problem. When it displayed, use Debug + Break All to break into the debugger. Next, use Debug + Windows + Modules, locate the DLL in the list. Right-click it and select "Symbol Load Information". You get a list of all the directories where the debugger searched for your PDB file. Make sure it is present in one of them.

Fwiw, your Command Arguments setting isn't kosher. It should be "$(TargetPath)" ENTRY_POINT. Double quotes to avoid trouble with spaces in the path name, no comma.

score 2 · Accepted Answer · answered Dec 26 '13 at 17:52

In short: All problems arose due to debugging a 32Bit dll in a 64Bit environment.

As can be seen from original question, and side issues mentioned in comments, I had few problems here:

Debugger is not present in a process it has allegedly launched.
When manually attached, symbols were not loaded.
When hitting "Break All" process apeared to be deadloack.

The reason is as mention the dll being 32Bit while debugger is 64Bit. The path to rundll32.exe interpernted as the 64Bit version. That normally causes WOW64 to launch a sub-process of the 32Bit version - hereby different process thus debugger not present.

Thanks all.

Debugging a native dll using rundll32.exe , fail to load symbols

2 Answers2