2

I have an old Visual Studio EEAddin-style debugger extension that is designed to take strings that have been mapped to integers at runtime and reverse the mapping in order to display the original string value in the watch window.

For example the display of the string mapped to the integer 0x39ad1dc4 is 0x39ad1dc4 (Hello, world!).

I would like to respect the setting of the debugger's "Hexadecimal Display" flag when I format the integer portion of the result, but I am not sure how (or even if) I can get access to that setting from my callback.

For reference, the EEAddin "custom viewer" callback I am using is:

typedef HRESULT (WINAPI * CUSTOMVIEWER)(DWORD address, DEBUGHELPER * helper, int base, BOOL useUnicodeStrings, char * result, size_t max, DWORD reserved);

One might think the base parameter would give me the information I need, but in my tests it's always been 10, no matter what the state of the hex-display option in the debugger is set to.

I have access to a DEBUGHELPER API that looks like this:

typedef struct tagDEBUGHELPER {
    DWORD dwVersion;

    HRESULT (WINAPI * ReadDebuggeeMemory)(struct tagDEBUGHELPER * self, DWORD address, DWORD bytesToRead, VOID * destination, DWORD * bytesActuallyRead );

    // The API calls below are only supported when dwVersion is >= 0x20000.
    DWORDLONG (WINAPI * GetRealAddress)(struct tagDEBUGHELPER * self);
    HRESULT (WINAPI * ReadDebuggeeMemoryEx)(struct tagDEBUGHELPER * self, DWORDLONG address, DWORD bytesToRead, VOID * destination, DWORD * bytesActuallyRead);

    int (WINAPI * GetProcessorType)(struct tagDEBUGHELPER * self);
} DEBUGHELPER;

I had to manually define this API based on sample code scattered around the internet; documentation of it and any potentially-related APIs is fairly hard to come by. The above structure and the callback type are the only portions of the API I'm aware of.

Is there a way I determine the state of the hexadecimal display checkbox with this, or some related API? I could re-write the viewer in a new extension platform if necessary, so long as that platform allows me to author extensions for native code.

  • What version of Visual Studio are you aiming now ? Do you use a .natvis file or autoexp.dat? You could give Format Specifiers a try.http://msdn.microsoft.com/en-us/library/75w45ekt.aspx – Lectem Oct 19 '14 at 10:50
  • It seems that the nBase argument was indeed the good thing to use according to http://flylib.com/books/en/4.441.1.51/1/ You might want to report it on MSDN – Lectem Oct 19 '14 at 11:20

0 Answers0