Do you use Visual Studio? If so, you can attach VS to any running process using the Debug | Attach To Process menu items. You can then break into the process and start examining stacks, threads, modules, etc.
If you want to delve deeper, you could download the Windows SDK and install the Debugging tools. This will give you KD and WinDBG - a console debugger and slightly more friendly multi-pane MDI-style debugging app respectively. Using these tools you can access to most of the core debugging infrastructure built into Windows.
However, note that this is not for the feint of heart and will require considerable time and effort to master. To really become a debugging guru, you'll also need to deeply understand the architecture of the kernel & OS and many core OS data structures.
Thus you might find the following books useful:
For .NET:
For Windows and/or .NET:
For Advanced Windows internals debugging
Enjoy! :)