2

In our c# code base (hence managed code), we have a class that we make extensive use of throughout the code. Given its ubiquity, I decided to write a custom debugger visualizer so that we could easily examine such objects when debugging. But, I hit a snag - when I try to run the visualizer in the IDE, I get a BadImageFormatException.

I am posting this to help others who come across the same error. I know what the issue and solution is and will post.

David I. McIntosh
  • 2,038
  • 4
  • 23
  • 45
  • You might want to ponder a bit why Microsoft's visualizers don't have that problem. You are doing it wrong, the question is not good enough to tell how. We can't even tell if this is a visualizer for C++ or managed code. Guessing at the latter, the solution platform is irrelevant, only the Project > Properties > Build settings matter. – Hans Passant Mar 04 '19 at 16:10
  • 1
    @hans-passant Would love to know then how to do it correctly when the build target on the application being built is x64. Can you point me somewhere? Thanks. – David I. McIntosh Mar 06 '19 at 23:56

2 Answers2

1

Currently (as of Visual Studio 2019) it's possible to split the visualizer into two:

  1. debuggee-side DLL -- gets injected into the target process, and
  2. debugger-side DLL -- loaded into Visual Studio.

The two halves pass data between each other using serialization/deserialization.

This architecture is required for visualizers to target multiple frameworks -- the debugger side is loaded into Visual Studio, so it must target .NET Framework; the debuggee side is injected into the target process, which might target .NET Core or .NET 5+. (I refer you to this repo for a minimal visualizer with this structure; and to other visualizers I've written (1 2) which also use a similar architecture.)

The same architecture works for bit-ness. Visual Studio is a 32-bit application, so the debugger side cannot be 64-bit; it must be 32-bit or AnyCPU. But if the target process might be 64-bit, the debuggee side has to match the target process and must be 64-bit or AnyCPU.

Per the docs:

Typically, it is best if both the debugger-side DLL and the debuggee-side DLL specify Any CPU as the target platform. The debugger-side DLL must be either Any CPU or 32-bit. The target platform for the debuggee-side DLL should correspond to the debugee process.

Zev Spitz
  • 13,950
  • 6
  • 64
  • 136
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/231344/discussion-between-zev-spitz-and-david-i-mcintosh). – Zev Spitz Apr 19 '21 at 22:10
  • The visualizers which Zev Spitz references above are well worth checking out. Thanks for these Zev! There are also other references with lots of information. – David I. McIntosh Apr 20 '21 at 14:29
0

The issue is that Visual Studio itself, the IDE, runs as a 32-bit process only. If it is to run a custom data visualizer for you while debugging, the custom visualizer and all the code that this visualizer loads must be loadable and runnable by a 32-bit process. The custom visualizer gets the object to visualize by a serialization/deserialization process. To deserialize the object, the visualizer must be able to load the .dll in which the object is defined. And here we run in to the snag: if we are building our application to an x64 target (rather than an AnyCpu target), we’re up a creek – it doesn’t matter if the custom visualizer itself is built to a 32-bit target, because it’s the application code that must be used for deserialization.

So if your application is built to a 64-bit target, you cannot run a custom visualizer (big, big OUCH Microsoft!). To get around the snag, you can build to a target of AnyCpu, and then things work well: the application loads and runs as 64-bit (since targeted to AnyCpu), but the IDE is still able to load the .dll’s as 32-bit for the purposes of the custom data visualizer running in the IDE’s process space.

If I am wrong on this and there is a better work-around, I would love to be corrected! Thanks.

David I. McIntosh
  • 2,038
  • 4
  • 23
  • 45