C# compiler (csc.exe) memory overflow compiling nested types and Linq

Question

I found one unpleasant behavior developing with Visual Studio. It was hanging my machine while compiling C#.

I have reduced behavior to the next minimal source code

using System.Collections.Generic;
using System.Linq;

namespace memoryOverflowCsharpCompiler {

    class SomeType { public decimal x; }

    class TypeWrapper : Dictionary<int,
                        Dictionary<int,
                        Dictionary<int, SomeType [] []>>> {

        public decimal minimumX() {
            return base.Values.Min(a =>
                      a.Values.Min(b =>
                      b.Values.Min(c =>
                      c       .Sum(d =>
                      d       .Sum(e => e.x)))));
        }
    }
}

Compiling with

PROMPT> csc source.cs

    *** BANG! overflow memory usage (up to ~3G)

PROMPT> csc /?
Microsoft (R) Visual C# Compiler version 12.0.30501.0
Copyright (C) Microsoft Corporation. All rights reserved.
...

(using Windows 8.1 Pro N x64; csc compiler process is running with 32bits)

sightly modifications don't produce this behavior (eg. changing decimal by int, reducing one nested level, ...), performing a big Select then reducing, works fine

Explicit workaround:

            return base.Values.SelectMany(a =>
                      a.Values.SelectMany(b =>
                      b.Values.Select    (c =>
                      c.       Sum       (d =>
                      d.       Sum       (e => e.x))))).Min();

Although this explicit workaround exists, it not guaranteed that this behavior will not occur again.

What's wrong?

Thank you!

Out of interest, have you tried this with the VS2015 preview? It would be interesting to know if it's fixed in Roslyn. — Jon Skeet, Dec 17 '14 at 10:31
This code has several things that make it hard to infer the types: 1) Implicit conversions 2) Many overloads 3) nested lambdas. If you try hard enough, you can encode NP complete SAT problems into these, so the compiler can't solve all overload resolution+type inferrence problems efficiently. But I don't know why your example is so bad. — CodesInChaos, Dec 17 '14 at 10:34
@JonSkeet Any volunteers? (I can't) I have searched about that but not found :( — josejuan, Dec 17 '14 at 10:35
@CodesInChaos but changing `decimal` by `int` works (with same level of 1), 2) and 3) — josejuan, Dec 17 '14 at 10:36
It seems that generic type resolution fails in that case. Specifying type argument explicitly fixes compilation problem — Krzysztof, Dec 17 '14 at 11:16
BTW on x64 I was able to compile it normally (but memory usage was around 2,5GB), but nesting it one more time caused overflow. — Krzysztof, Dec 17 '14 at 11:23

score 3 · Accepted Answer · answered Dec 17 '14 at 11:42

It seems that generic type resolution fails in that case. Changing from decimal to int works by chance. If you increase nesting level, than you'll see that it also fails for int. On my x64 machine this code compiles for both int and decimal and uses around 2.5GB of memory, but increasing nesting level results in overflow when memory usage growing to aroung 4GB.

Specifying type argument explicitly allows to compile code:

class TypeWrapper : Dictionary<int, Dictionary<int, Dictionary<int, Dictionary<int, SomeType[][]>>>>
{
    public decimal minimumX()
    {
        return base.Values
            .Min<Dictionary<int, Dictionary<int, Dictionary<int, SomeType[][]>>>, decimal>(a => a.Values
                .Min<Dictionary<int, Dictionary<int, SomeType[][]>>, decimal>(b => b.Values
                    .Min<Dictionary<int, SomeType[][]>, decimal>(c => c.Values
                        .Min(d => d
                            .Sum(e => e.Sum(f => f.x))
                        )
                    )
                )
            );
    }
}

Also compiler works when you reduce nesting by introducing local variable:

class TypeWrapper : Dictionary<int, Dictionary<int, Dictionary<int, Dictionary<int, SomeType[][]>>>>
{
    public decimal minimumX()
    {
        Func<Dictionary<int, SomeType[][]>, decimal> inner = (Dictionary<int, SomeType[][]> c) => c.Values
                        .Min(d => d
                            .Sum(e => e.Sum(f => f.x))
                        );

        return base.Values
            .Min(a => a.Values
                .Min(b => b.Values
                    .Min(inner)
                )
            );
    }
}

C# compiler (csc.exe) memory overflow compiling nested types and Linq

1 Answers1