What worked in previous version was to activate the line
<Define name="No_ExceptionsWindow"/>
in the ApplicationCommands.vsct
included in the solution created by Visual Studio. This disabled the menu item "Debug -> Exceptions".
The line (in a disabled commented-out was) is still created in the newer Visual Studio versions but doesn't work any more. This is clearly a bug caused by Microsoft when they implemented the new cool Exception Settings tool window. Either they completely forgot to include the checking for the "No_ExceptionsWindow" define or they renamed the define without adapting the Visual Studio Solution Template for VS Shell projects.
I would contact the Visual Studio Team at Microsoft directly. They might fix this bug in future updates or they might be able to tell you what string to define to disable the Exception Settings Tool Window.