4

I'm attempting to enumerate symbols from a DLL that I have loaded. For those interested, this is part of the CPPCoverage project, and for some functionality I need symbol data.

Breakdown of the problem

When the process is started or a DLL is loaded, symbols need to be enumerated for some of the new functionality that has been planned.

Basically, a process is created, and dbghelp is used to get symbol information. Next, symbols are iterated using SymEnumSymbols. There are two moments when this happens:

  1. When the process is started (CREATE_PROCESS_DEBUG_EVENT)
  2. When a DLL is loaded (LOAD_DLL_DEBUG_EVENT)

Everything works fine during (2). However, symbols cannot be enumerated during (1).

Behavior is that everything works fine, until the SymEnumSymbols call. The return value tells me there's an error, but GetLastError returns SUCCESS. Also, the callback function isn't called.

To make it even more weird, a call to SymGetSymFromName does actually work.

Minimal test case

static BOOL CALLBACK EnumerateSymbols(
                          PSYMBOL_INFO pSymInfo, ULONG SymbolSize, PVOID UserContext)
{
    std::cout << "Symbol: " << pSymInfo->Name << std::endl;
    return TRUE;
}

void Test()
{
    SymSetOptions(SYMOPT_LOAD_ANYTHING);

    STARTUPINFO si;
    PROCESS_INFORMATION pi;
    ZeroMemory(&si, sizeof(si));
    si.cb = sizeof(si);
    ZeroMemory(&pi, sizeof(pi));

    auto str = "FullPathToSomeExeWithPDB.exe";
    auto result = CreateProcess(str, NULL, NULL, NULL, FALSE,
                                DEBUG_PROCESS, NULL, NULL, &si, &pi);
    if (result == 0)
    {
        auto err = GetLastError();
        std::cout << "Error running process: " << err << std::endl;
        return;
    }

    if (!SymInitialize(pi.hProcess, NULL, FALSE))
    {
        auto err = GetLastError();
        std::cout << "Symbol initialization failed: " << err << std::endl;
        return;
    }

    bool first = false;
    DEBUG_EVENT debugEvent = { 0 };
    while (!first)
    {
        if (!WaitForDebugEvent(&debugEvent, INFINITE))
        {
            auto err = GetLastError();
            std::cout << "Wait for debug event failed: " << err << std::endl;
            return;
        }
        if (debugEvent.dwDebugEventCode == CREATE_PROCESS_DEBUG_EVENT)
        {
            auto dllBase = SymLoadModuleEx(
                pi.hProcess,
                debugEvent.u.CreateProcessInfo.hFile,
                str,
                NULL,
                reinterpret_cast<DWORD64>(debugEvent.u.CreateProcessInfo.lpBaseOfImage),
                0,
                NULL,
                0);

            if (!dllBase)
            {
                auto err = GetLastError();
                std::cout << "Loading the module failed: " << err << std::endl;
                return;
            }

            if (!SymEnumSymbols(pi.hProcess, dllBase, NULL, EnumerateSymbols, nullptr))
            {
                auto err = GetLastError();
                std::cout << "Error: " << err << std::endl;
            }

            first = true;
        }
    } 
    // cleanup code is omitted
}
atlaste
  • 30,418
  • 3
  • 57
  • 87
  • Ok. You **are** calling `GetLastError` too late. You need to call it **immediately** after the condition was met, where it is documented to return meaningful values. Do not intersperse it with **any** other code. – IInspectable Jun 30 '17 at 09:11
  • @IInspectable Test cases... normally I'd just throw an exception. Anyways, I just modified it to call it immediately. Doesn't change a thing... – atlaste Jun 30 '17 at 09:15
  • I suppose that [CREATE_PROCESS_DEBUG_EVENT](https://msdn.microsoft.com/en-us/library/windows/desktop/ms679302.aspx) is raised, before any modules have been loaded: *"The system generates this debugging event before the process begins to execute in user mode"*. [Why do I get ERROR_INVALID_HANDLE from GetModuleFileNameEx when I know the process handle is valid?](https://blogs.msdn.microsoft.com/oldnewthing/20150716-00/?p=45131) seems relevant. – IInspectable Jun 30 '17 at 09:49
  • @IInspectable - when `CREATE_PROCESS_DEBUG_EVENT` raised - exe and ntdll.dll already loaded – RbMm Jun 30 '17 at 09:57
  • @RbMm: Do you know, whether the user-mode module information tables have been initialized at that point? My thinking was, that while a process object has been created, and the bare minimum of sections are mapped into the address space, none of the user-mode bootstrapping code has run yet, leaving module queries to return empty result lists. – IInspectable Jun 30 '17 at 10:22
  • @IInspectable Not sure, but if they aren't initialized, `SymGetSymFromName` should fail too right? That call works just fine. – atlaste Jun 30 '17 at 11:01
  • *"Initialized"* was the wrong term. *"Populated"* is probably more to the point. – IInspectable Jun 30 '17 at 11:06
  • but `SymEnumSymbols` enumerate not modules. it enumerate symbols in concrete module – RbMm Jun 30 '17 at 11:07
  • @IInspectable To be sure, I just hacked some code in my real project that executes the same code on the first breakpoint in the coverage tool instead of during the `CREATE_PROCESS_DEBUG_EVENT`. Unfortunately the behavior is the same, so I suppose that's not it. – atlaste Jun 30 '17 at 11:44
  • faster of all `SymLoadModuleEx` fail find pdb file for your exe. for ensure in this call `SymGetModuleInfo64` after `SymLoadModuleEx` and check `IMAGEHLP_MODULE64.SymType` - i guess that `SymNone` will be here. but you need `SymPdb`. so may be you need call `SymSetSearchPath` at begin – RbMm Jun 30 '17 at 12:22
  • at all look in "FullPathToSomeExeWithPDB.exe" (simply search .pdb string in it) - are path is absolute or relative. if absolute - are .pdb file exist at this location, if relative - where is pdb file stored ? – RbMm Jun 30 '17 at 12:35
  • @RbMm just tested the `IMAGEHLP_MODULE64.SymType`: it's actually `SymPdb`. Paths are absolute in my case. – atlaste Jun 30 '17 at 12:36
  • why in this case simply not look under debugger, what is inside `SymEnumSymbols` happens ? – RbMm Jun 30 '17 at 12:47
  • @RbMm ehhh a lack of source code? – atlaste Jun 30 '17 at 13:30
  • the pdb for dbghelp is enough. when enum symbols is ok, usually call tree look like [`this`](https://prnt.sc/fq07kh) - you can try look at which point it break in your case. – RbMm Jun 30 '17 at 13:32
  • @RbMm Since I don't have VS2017, I tried stepping through the application and putting breakpoints on calls. From what I can tell, I get the same call tree as in your image. – atlaste Jul 03 '17 at 06:31
  • I can't reproduce. Your code works fine for me. I've tested it on a simple C# console app .exe, and it does display "Main". – Simon Mourier Jul 03 '17 at 09:10
  • @SimonMourier Huh? That's weird (but valuable input)... I'll test it on another pc, let's see what happens... As for my test setup: it's x64, the file I'm using for the test is the minimumtestapp which can be found here: https://github.com/atlaste/CPPCoverage/tree/master/MinimumTestApp . The test code as-is is currently even commented out as part of the coverage code: https://github.com/atlaste/CPPCoverage/blob/master/Coverage/Main.cpp . – atlaste Jul 03 '17 at 09:13

2 Answers2

4

Brr, quite a stumper. I got a repro for this in VS2017, using a simple do-nothing target executable built from the Win32 Console project template. Nothing I tried could convince SymEnumSymbols() to enumerate any symbols. I next expanded on the code, also trapping the LOAD_DLL_DEBUG_EVENT notification:

if (debugEvent.dwDebugEventCode == LOAD_DLL_DEBUG_EVENT) {
    auto base = SymLoadModule64(pi.hProcess, debugEvent.u.LoadDll.hFile, NULL, NULL, NULL, 0);
    if (!base) {
        auto err = GetLastError();
        std::cout << err << std::endl;
     }
    else {
        CloseHandle(debugEvent.u.LoadDll.hFile);
        SymEnumSymbols(pi.hProcess, base, NULL, EnumerateSymbols, nullptr);
    }
}

Along with setting the symbol search path correctly in SymInitialize(), that worked just fine and properly listed the symbols in ntdll.dll etc.

Conclusion: there is something wrong with the PDB file

That paid-off. Microsoft has been tinkering with the PDB file generation, starting in VS2015. They added the /DEBUG:FASTLINK option. Note that the linked docs are misleading, it is also the default in VS2015. The resulting PDB file cannot be properly enumerated by the operating system's version of DbgHelp.dll. The GetLastError() code was quite misleading and I spent entirely too much time on it, I think it merely indicates "I successfully enumerated nothing". Note how this code is documented for other DbgHelp api functions like SymSetContext and SymLoadModuleEx.

In VS2015 use Project > Properties > Linker > Debug > Generate Debug Info = "Optimize for debugging (/DEBUG)".

In VS2017 use Project > Properties > Linker > Debug > Generate Debug Info = "Generate Debug Information optimized for sharing and publishing (/DEBUG:FULL)".

Emphasizing that these setting matter on the target project, not the debugger project. Ideally there would a DbgHelp.dll version that could read debug info from the fastlink version of the PDB as well. I could not find one, the ones that came along with SDK 8.1 and SDK 10 did not solve the problem. Yet another case of the DevDiv and Windows groups not working together.

Hans Passant
  • 922,412
  • 146
  • 1,693
  • 2,536
  • Thanks. Moments ago I came to the same conclusion. What strikes me is that VS2017 doesn't support it as well - I hoped this would be fixed by now. The post from 25-02 that I linked above states "The latest VS 2017 pre-release would install a new dbghelp.dll under VS installation directory that works with fastlink PDBs." -- so I guess that's that for now... – atlaste Jul 03 '17 at 13:05
  • 1
    That's just not true, no new dbghelp.dll. YongKang Zhu pretty much acknowledges this by noting that DIA can't enumerate either. I did not document that picking a specific symbol worked, passing "main" for the 3rd argument instead of NULL avoided failure. – Hans Passant Jul 03 '17 at 13:14
1

After the comment from @SimonMournier I ran a lot of other tests. Eventually, I was able to figure out what the issue here is. As it turns out, the linker flag /DEBUG:FastLink in Visual Studio actually causes the issue.

After some google'ing I found this notice on the community forums: https://developercommunity.visualstudio.com/content/problem/4631/dia-sdk-still-doesnt-support-debugfastlink.html

[...] Windows debuggers team has been informed to build a new dbghelp.dll with VS 2017 PDB/DIA static libraries and the next public release of Windows SDK (or debugger kits) will contain dbghelp.dll that is able to deal with fastlink PDBs. The latest VS 2017 pre-release would install a new dbghelp.dll under VS installation directory that works with fastlink PDBs.

So, in short, it simply won't work with Visual Studio 2015, because DIA simply doesn't support it. When we're upgrading to VS2017, it'll be automatically fixed. Also, when linking with /DEBUG , everything will work out fine.

atlaste
  • 30,418
  • 3
  • 57
  • 87