C# code very slow with debugger attached; MemoryMappedFile's fault?

Question

I have a client/server app. The server component runs, uses WCF in a 'remoting' fashion (binary formatter, session objects).

If I start the server component and launch the client, the first task the server does completes in <0.5sec.

If I start the server component with VS debugger attached, and then launch the client, the task takes upwards of 20sec to complete.

There are no code changes - no conditional compilation changes. The same occurs whether I have the server component compiled and running in 32-bit, 64-bit, with the VS hosting process, without the VS hosting process, or any combination of those things.

Possibly important: If I use the VS.NET profiler (sampling mode), then the app runs as quick as if there were no debugger attached. So I can't diagnose it that way. Just checked, instrumentation mode also runs quickly. Same for the concurrency profiling mode, works quickly.

Key data:

• The app uses fairly heavy multithreading (40 threads in the standard thread pool). Creating the threads happens quickly regardless and is not a slow point. There are many locks, WaitHandles and Monitor patterns
• The app raises no exceptions at all.
• The app creates no console output.
• The app is entirely managed code.
• The app does map a few files on disk to a MemoryMappedFile: 1x750MB and 12x8MB and a few smaller ones

Measured performance:

• CPU use is minimal in both cases; when debugger is attached, CPU sits at <1%
• Memory use is minimal in both cases; maybe 50 or 60MB in both cases
• There are plenty of page faults happening (ref MMF), however they happen more slowly when the debugger is attached
• If the VS hosting process is not used, or basically the 'remote debugging monitor' comes into play, then that uses a decent amount CPU and creates a good number of page faults. But that's not the only time the problem is occurring
• The performance difference is seen regardless of how the client is run. The only variable being changed is the server component being run via 'Start with debugging' vs launched from Explorer.

My ideas:

• WCF slow when debugged?
• MemoryMappedFiles slow when debugged?
• 40 threads used - slow to debug? Perhaps Monitors/locks notify debugger? Thread scheduling becomes strange/context switches very infrequent?
• Cosmic background radiation granting intelligence and cruel sense of humour to VS

All seem stupidly unlikely.

So, my questions:

1. Why is this happening?
2. If #1 unknown, how can I diagnose / find out?

Answer 1

Since this is one of the first results when googling for this issue I would like to add my problem solution here in the hopes of saving someone 2 hours of research like in my case.

My code slowed down from 30 seconds without debugger attached to 4 minutes with debugger. because I forgot to remove a conditional breakpoint. These seem to slow down execution tremendously, so watch out for those

Answer 2

Exceptions can notably impact the performance of an application. There are two types of exceptions: 1st chance exceptions (the one gracefully handled with a try/catch block), and unhandled exceptions (that will eventually crash the application).

By default, the debugger does not show 1st chance exceptions, it just shows unhandled exceptions. And by default, it also shows only exceptions occurring in your code. However, even if it does not show them, it still handles them, so its performance may be impacted (especially in load tests, or big loop runs).

To enable 1st chance exceptions display in Visual Studio, click on "Debug | Exceptions" to invoke the Exceptions dialog, and check "Thrown" on the "Common language runtime" section (you can be more specific and choose wich 1st chance exception you want to see).

To enable 1st chance exceptions display originating from anywhere in the application, not just from your code, click on "Tools | Options | Debugging | General" and disable the "Enable Just My Code" option.

And for these specific "forensics mode" cases, I also strongly recommend to enable .NET Framework Source Stepping (it requires "Enable Just My Code" to be disabled). It's very useful to understand what's going on, sometimes just looking at the call stack is very inspiring - and helpful especially in the case of cosmic radiation mixup :-)

Two related interesting articles:

• How to debug crashes and hangs
• Configuring Visual Studio to Debug .NET Framework Source Code

Answer 3

Possible causes:

1. Various special kinds of breakpoints such as:

   • Conditional breakpoints
   • Memory changed breakpoints
   • Function breakpoints

2. Having the "Enable native code debugging" option checked.

   • This option makes debug runs slow as molasses.
   • This option is not under Tools -> Options -> Debug, (that would make too much sense,) it is under Project -> Properties -> Debug

3. Excessive use of System.Diagnostics.Debug.Write().

   • My benchmarks show that 1000 invocations of Debug.WriteLine() take only 10 milliseconds when running without debugging, but a whole 500 milliseconds when debugging. Apparently the Visual Studio Debugger, when active, intercepts DotNet debug output, and does extremely time-consuming stuff with it. (Over decades of using Microsoft products, we have come to expect nothing less from Microsoft.)
   • Replacing Debug.WriteLine() with kernel32.dll -> PInvoke -> OutputDebugStringW() does not help, because when running a DotNet application, Visual Studio completely ignores kernel32 debug output and only displays DotNet debug output, which is a completely different thing. (And I suppose that anything else would, again, make too much sense.)

4. Excessive amount of exceptions being thrown and caught, as another answer suggests.

   • Throwing an exception under DotNet is a mind-bogglingly slow operation.
   • Collecting a stack trace under DotNet is an insanely slow operation.