8

On the project I am working on at the moment we have an unknown number of flickering tests which fail at varying rates (anywhere from every-other run to once every 5000 runs) which has led us to believe we have some serious Heisenbugs to resolve. Most of the time including trace statements in the code change the timing enough to eliminate these errors, or at least make them occur less often, but obviously this is not a valid bug fix.

We are wondering if there are any tools similair to CHESS (http://research.microsoft.com/en-us/projects/chess/download.aspx) that work in/with Visual Studio 2010 to stress test applications and detect potential race conditions and deadlocks?

Pete Martin
  • 856
  • 7
  • 15
  • 1
    Following SO question might help: http://stackoverflow.com/questions/1715822/unit-test-for-thread-safe-ness – Christophe Geers Sep 15 '11 at 08:52
  • 1
    Thanks - come useful information there, especially on writing threadsafe tests and Typemock Racer. The information here (http://stackoverflow.com/questions/537014/using-tdd-to-drive-out-thread-safe-code) was also useful. – Pete Martin Sep 15 '11 at 09:28
  • Any chance you are trying to *unit test* classes that deal with singletons and static objects which are written to in some *unit test* and read in other? – Darin Dimitrov Sep 15 '11 at 09:39
  • We're only using static objects in the plumbing for our tests in SetUp() methods, but we've rigorously reset them in our TearDown() methods and they never come under test. It is a very large project with a lot of concurrency, we know the problems lie in our production code, we just need to be able to consistently reproduce the race/deadlock conditions to find the problems - we are also convinced that there are more concurrency issues than we have actually witnessed so far. A tool like CHESS would solve all this for us. – Pete Martin Sep 15 '11 at 09:59

2 Answers2

3

Within Visual Studio 2010 (not sure which editions exactly) you can create a new profiling session with the "Concurrency" option. This should detect the deadlocks you're looking for. You can then profile a unit test to automate your many runs.

Couldn't find many images of the results but here's an example example
(source: microsoft.com)

Glorfindel
  • 21,988
  • 13
  • 81
  • 109
Ian
  • 33,605
  • 26
  • 118
  • 198
  • And unfortunately we don't have Ultimate :( – Pete Martin Sep 15 '11 at 12:58
  • 5
    http://msdn.microsoft.com/en-us/library/dd537632.aspx http://msdn.microsoft.com/en-us/magazine/ee336027.aspx This is documentation on how to use VS2010 Concurrency Visualizer - For the benefit of stackoverflow users – Bhuvan Nov 09 '11 at 18:48
2

Another tool is from Microsoft Research only, you can download it from Codeplex - chesstool.codeplex.com. It supports .Net 4.0 libraries, next version of Chess, but I don't know you are able to use it.

Anuraj
  • 18,859
  • 7
  • 53
  • 79