15

I'm getting an unexpected NullReferenceException when I run this code, omitting the fileSystemHelper parameter (and therefore defaulting it to null):

public class GitLog
    {
    FileSystemHelper fileSystem;

    /// <summary>
    ///   Initializes a new instance of the <see cref="GitLog" /> class.
    /// </summary>
    /// <param name="pathToWorkingCopy">The path to a Git working copy.</param>
    /// <param name="fileSystemHelper">A helper class that provides file system services (optional).</param>
    /// <exception cref="ArgumentException">Thrown if the path is invalid.</exception>
    /// <exception cref="InvalidOperationException">Thrown if there is no Git repository at the specified path.</exception>
    public GitLog(string pathToWorkingCopy, FileSystemHelper fileSystemHelper = null)
        {
        this.fileSystem = fileSystemHelper ?? new FileSystemHelper();
        string fullPath = fileSystem.GetFullPath(pathToWorkingCopy); // ArgumentException if path invalid.
        if (!fileSystem.DirectoryExists(fullPath))
            throw new ArgumentException("The specified working copy directory does not exist.");
        GitWorkingCopyPath = pathToWorkingCopy;
        string git = fileSystem.PathCombine(fullPath, ".git");
        if (!fileSystem.DirectoryExists(git))
            {
            throw new InvalidOperationException(
                "There does not appear to be a Git repository at the specified location.");
            }
        }

When I single step the code in the debugger, after I step over the first line (with the ?? operator) then fileSystem still has the value null, as shown in this screen snip (stepping over the next line throws NullReferenceException): When is null not null?

This is not what I expected! I'm expecting the null coalescing operator to spot that the parameter is null and create a new FileSystemHelper(). I have stared at this code for ages and can't see what is wrong with it.

ReSharper pointed out that the field is only used in this one method, so could potentially be converted to a local variable... so I tried that and guess what? It worked. So, I have my fix, but I cannot for the life of me see why the code above doesn't work. I feel like I am on the edge of learning something interesting about C#, either that or I've done something really dumb. Can anyone see what's happening here?

Taylan Aydinli
  • 4,333
  • 15
  • 39
  • 33
Tim Long
  • 13,508
  • 19
  • 79
  • 147
  • You are already stating that `fileSystemHelper` is `null` in the method parameters, I'm not sure, but it might have something to do with it. But then again, I'm guessing. – trinaldi Oct 14 '13 at 01:10
  • 2
    Are you sure the NRE doesn't occur from *within* GetFullPath (ignoring what the watch shows)? I see nothing with the above code that would result in said behavior. – user2864740 Oct 14 '13 at 01:14
  • OK, having quit from VisualStudio to do something else then reloaded it, everything is working now and I can't reproduce the problem. I _think_ this might be an odd caching issue with the ReSharper unit test runner. I'm using a simple MSpec test to exercise the code. ReSharper takes a shadow copy of the assembly when running unit tests and sometimes, just sometimes, the shadow copy seems to get 'stuck', I've seen it happen a couple of times before. So most likely, I was actually running old code, even though I had manually rebuilt everything. It's the best explanation I have... – Tim Long Oct 14 '13 at 02:00

2 Answers2

12

I have reproduced it in VS2012 with the following code:

public void Test()
{
    TestFoo();
}

private Foo _foo;

private void TestFoo(Foo foo = null)
{
    _foo = foo ?? new Foo();
}

public class Foo
{
}

If you set a breakpoint at the end of the TestFoo method, you would expect to see the _foo variable set, but it will still show as null in the debugger.

But if you then do anything with _foo, it then appears correctly. Even a simple assignment such as

_foo = foo ?? new Foo();
var f = _foo;

If you step through it, you'll see that _foo shows null until it is assigned to f.

This reminds me of deferred execution behavior, such as with LINQ, but I can't find anything that would confirm that.

It's entirely possible that this is just a quirk of the debugger. Perhaps someone with MSIL skills can shed some light on what is happening under the hood.

Also interesting is that if you replace the null coalescing operator with it's equivalent:

_foo = foo != null ? foo : new Foo();

Then it does not exhibit this behavior.

I am not an assembly/MSIL guy, but just taking a look at the dissasembly output between the two versions is interesting:

        _foo = foo ?? new Foo();
0000002d  mov         rax,qword ptr [rsp+68h] 
00000032  mov         qword ptr [rsp+28h],rax 
00000037  mov         rax,qword ptr [rsp+60h] 
0000003c  mov         qword ptr [rsp+30h],rax 
00000041  cmp         qword ptr [rsp+68h],0 
00000047  jne         0000000000000078 
00000049  lea         rcx,[FFFE23B8h] 
00000050  call        000000005F2E8220 
        var f = _foo;
00000055  mov         qword ptr [rsp+38h],rax 
0000005a  mov         rax,qword ptr [rsp+38h] 
0000005f  mov         qword ptr [rsp+40h],rax 
00000064  mov         rcx,qword ptr [rsp+40h] 
00000069  call        FFFFFFFFFFFCA000 
0000006e  mov         r11,qword ptr [rsp+40h] 
00000073  mov         qword ptr [rsp+28h],r11 
00000078  mov         rcx,qword ptr [rsp+30h] 
0000007d  add         rcx,8 
00000081  mov         rdx,qword ptr [rsp+28h] 
00000086  call        000000005F2E72A0 
0000008b  mov         rax,qword ptr [rsp+60h] 
00000090  mov         rax,qword ptr [rax+8] 
00000094  mov         qword ptr [rsp+20h],rax 

Compare that to the inlined-if version:

        _foo = foo != null ? foo : new Foo();
0000002d  mov         rax,qword ptr [rsp+50h] 
00000032  mov         qword ptr [rsp+28h],rax 
00000037  cmp         qword ptr [rsp+58h],0 
0000003d  jne         0000000000000066 
0000003f  lea         rcx,[FFFE23B8h] 
00000046  call        000000005F2E8220 
0000004b  mov         qword ptr [rsp+30h],rax 
00000050  mov         rax,qword ptr [rsp+30h] 
00000055  mov         qword ptr [rsp+38h],rax 
0000005a  mov         rcx,qword ptr [rsp+38h] 
0000005f  call        FFFFFFFFFFFCA000 
00000064  jmp         0000000000000070 
00000066  mov         rax,qword ptr [rsp+58h] 
0000006b  mov         qword ptr [rsp+38h],rax 
00000070  nop 
00000071  mov         rcx,qword ptr [rsp+28h] 
00000076  add         rcx,8 
0000007a  mov         rdx,qword ptr [rsp+38h] 
0000007f  call        000000005F2E72A0 
        var f = _foo;
00000084  mov         rax,qword ptr [rsp+50h] 
00000089  mov         rax,qword ptr [rax+8] 
0000008d  mov         qword ptr [rsp+20h],rax 

Based on this, I do think there is some kind of deferred execution happening. The assignment statement in the second example is very small in comparison to the first example.

Matt Johnson-Pint
  • 230,703
  • 74
  • 448
  • 575
  • 3
    This appears to be a problem with the 64-bit debugger. Building and compiling targeting "Any CPU" exhibits this behavior. Changing it to target x86, then it is no longer a problem. – Jeff Mercado Oct 14 '13 at 01:59
  • 3
    I think I know what the problem is... the line numbers generated by VS in the 64-bit build is incorrect. The line number generated for the line following the null-coalescing line is actually "in the middle" of the null-coalescing instruction. So the debugger breaks on that line but the previous instruction hasn't fully completed yet. Stepping over that line will finish that instruction. You can see the point at which the member variable gets set if you step through the disassembly. – Jeff Mercado Oct 14 '13 at 02:12
  • @JeffMercado - Confirmed. It only does this with "Any CPU" or "x64". Works fine with "x86". – Matt Johnson-Pint Oct 14 '13 at 02:12
  • This is eerily similar to [this](http://geekswithblogs.net/twickers/archive/2011/03/31/misreporting-of-variable-values-when-debugging-x64-code-with-the.aspx) and [this](http://connect.microsoft.com/VisualStudio/feedback/details/655793/). But those were supposed to have been fixed already. Perhaps a regression issue? – Matt Johnson-Pint Oct 14 '13 at 02:22
  • 1
    I have opened a bug on Connect, using the above repro source code and linking to this question. If you can reproduce the issue then you could visit the issue report and click on the "I can too" where it says "X users can reproduce this" and/or vote it up. Its hardly a show-stopper but it does cause a lot of confusion and wasted time. https://connect.microsoft.com/VisualStudio/feedback/details/805334/x64-compiler-generates-incorrect-source-lines-for-null-coalescing-operator-resulting-in-incorrect-value-display-during-debugging – Tim Long Oct 14 '13 at 14:31
  • 1
    Is "it's working in RyuJIT" an acceptable resolution to this issue? – Kevin Frei Feb 10 '14 at 23:00
  • BTW: I already reproduced the problem (trivial repro: I love the StackOverflow community's willingness to create simplified repros!), but we're trying to end-of-life the current x64 JIT, so I'd rather just get RyuJIT finished up for x64. This issue doesn't occur with current RyuJIT bits (coming soon to a preview release near you!) – Kevin Frei Feb 11 '14 at 17:38
1

Someone else experienced the same problem in this question. Interestingly it is also using a this._field = expression ?? new ClassName(); format. It could be some kind of issue with the debugger, as writing the value out seemed to produce the correct results for them.

Try adding debug/log code to show the value of the field after assignment to eliminate weirdness in the attached debugger.

Community
  • 1
  • 1
Timothy Walters
  • 16,866
  • 2
  • 41
  • 49