Can Pex instrumentation cause code to change its effect?

Question

I've got a simple JSON lexer class; it takes a string and generates an IJSONValue; the IJSONValue has a ToJSONString method that returns a valid JSON string.

The code for that, of course, is quite complicated with plenty of branches; that's why I thought this would be the perfect place to test Pex's abilities. I created the following test:

[TestClass]
[PexClass]
public partial class JSONTests {
    [PexGenericArguments(typeof(JSONArray))]
    [PexGenericArguments(typeof(JSONBoolean))]
    [PexGenericArguments(typeof(JSONNull))]
    [PexGenericArguments(typeof(JSONNumber))]
    [PexGenericArguments(typeof(JSONObject))]
    [PexGenericArguments(typeof(JSONString))]
    [PexMethod]
    public void TestLexer<T>([PexAssumeNotNull] T value) where T : IJSONValue {
        string json = value.ToJSONString();
        IJSONValue result = new JSONLexer().GetValue(json);

        PexAssert.AreEqual(value, result);
    }
}

Running Pex on this I found a few non-issues with null handling, which I had fixed. However, I also got a lot of methods reporting exceptions that didn't make sense. They look like this:

[TestMethod]
[PexGeneratedBy(typeof(JSONTests))]
[PexRaisedException(typeof(JSONException))]
public void TestLexerThrowsJSONException78() {
    JSONBoolean s0 = new JSONBoolean(true);
    this.TestLexer<JSONBoolean>(s0);
}

This, however, is pretty similar to one of my tests which I know works. I ran it in the debugger, and outside the debugger, and in both cases the test passed. The most boggling thing is that the exception text actually makes some sense; it's the text that would have been reported if the Constant regex didn't match the string "false". I get similar exceptions for other regex mismatches, which don't make sense.

Why does Pex think this throws an Exception? Does the instrumentation mess with ThreadLocal or Regex in a weird way? This is what my regex-holding class looks like (regexes were redacted for brevity).

private static class Regexes {
    private static RegexOptions Options = RegexOptions.Compiled | RegexOptions.IgnorePatternWhitespace;
    public static ThreadLocal<Regex> String = new ThreadLocal<Regex>(() => new Regex(@"(...)", Options));
    public static ThreadLocal<Regex> Number = new ThreadLocal<Regex>(() => new Regex(@"(...)", Options));
    public static ThreadLocal<Regex> Constant = new ThreadLocal<Regex>(() => new Regex(@"(...)", Options));
}

Can Pex instrumentation cause code to change its effect?

0 Answers0