18

I wanted to try to allocate a 4 billion bytes array and this is my C# code:

long size = 4 * 1000;
size *= 1000;
size *= 1000;
byte[] array = new byte[size];

this code fails with System.OverflowException on the line containing new. Okay, turns out Length returns int, so the array length is also limited to what int can store.

Then why is there no compile-time error and long is allowed to be used as the number of array elements at allocation?

Ardahan Kisbet
  • 641
  • 1
  • 13
  • 33
sharptooth
  • 167,383
  • 100
  • 513
  • 979
  • 6
    `Array.CreateInstance()` gives the more meaningful error message, "Arrays larger than 2GB are not supported." – Bradley Smith Jun 08 '12 at 08:34
  • So you're allocating 3.7 GB of memory. How much RAM is installed in your machine? :D – konqi Jun 08 '12 at 08:35
  • JoSo, the amount of physical memory is of no importance there. – Joey Jun 08 '12 at 08:36
  • 2
    @Joso: That's a 64-bit process so thanks to paging it can allocate 3.7 ih there's enough disk space. – sharptooth Jun 08 '12 at 08:36
  • 1
    Even paging has its limits. Assuming 1024 MB physical RAM, Windows would usually limit the paging file to 1536 MB. Combined you'd be able to allocate 2560 MB of memory. Since windows sets the upper page file size to 1.5 times of physical memory i'll say that physical RAM may not be the limiting factor but it is a factor. – konqi Jun 08 '12 at 08:44
  • 3
    JoSo, (a) no one in their right mind would use a 64-bit Windows on just a GiB of memory; (b) memory is dirt-cheap these days so anything less than 8 GiB makes little sense anyway; (c) have you read the answers? The problem is something else entirely. – Joey Jun 08 '12 at 08:47

5 Answers5

23

Because the specification says so in section 7.6.10.4:

Each expression in the expression list must be of type int, uint, long, or ulong, or implicitly convertible to one or more of these types.

This is most likely to easily allow creation of arrays larger than 2 GiB, even though they are not supported yet (but will be without a language change once the CLR makes such a change). Mono does support this, however and .NET 4.5 apparently will allow larger arrays too.

Regarding array length being an int by the way: There is also LongLength, returning a long. This was in .NET 1.1 and probably a future-proofing change.

Joey
  • 344,408
  • 85
  • 689
  • 683
  • 1
    +1 Good spot on the 4.5 stuff, hope to God I never see that turned on in our app... – Adam Houldsworth Jun 08 '12 at 08:42
  • 1
    Looks like even with the 4.5 change, the code in question will still fail because it says "The maximum index in any single dimension is 2,147,483,591 (0x7FFFFFC7) for byte arrays and arrays of single-byte structures, and 2,146,435,071 (0X7FEFFFFF) for other types." – Eren Ersönmez Jun 08 '12 at 10:42
  • Indeed. You could make it a multidimensional array to work around that, then. But that's nasty as well. – Joey Jun 08 '12 at 10:44
  • @ErenErsönmez you wrote "it says..." where does it say that? – Mishax Feb 15 '13 at 05:58
  • @Mishax: In the second link I included in my answer. – Joey Feb 15 '13 at 06:54
11

why long is allowed as array length?

Answer is: long in .net means Int64

And array indexing can be Int64 according to specification.

2nd question: Why overflowexception is showing?

Because any single object can not be allocated more than 2GB of memory.

IS4
  • 11,945
  • 2
  • 47
  • 86
Md Kamruzzaman Sarker
  • 2,387
  • 3
  • 22
  • 38
  • 3
    With such claims (unless they're obvious, e.g. `long` being an alias for `System.Int64`) it's always nice to back them up with references. – Joey Jun 08 '12 at 08:59
  • 1
    I disagree with your second point. If its out of memory, you should expect an OutOfMemoryException, not an OverflowException. According to the specification, an OverflowException should only be raised if the length is less than zero. – Mishax Feb 15 '13 at 06:01
  • 1
    @Mishax: It's not out of memory, it's just that a single object cannot be larger than that. – Joey Feb 15 '13 at 06:55
  • 1
    @Joey You may be right but I'm referring to mdkamruzzaman's explanation of the OverflowException. In the C# language specification it states that an OverflowException can be thrown if any dimension length specifier is less than zero. And that's it, that's the only reason. I can find no mention whatsoever in the language specification to a 2GB limit. There's some discussion here: http://stackoverflow.com/questions/14889333/array-creation-expressions-and-long-dimension-lengths – Mishax Feb 16 '13 at 05:49
7

It is a limitation of the CLR, no single object can exceed 2GB, including arrays:

Large array C# OutOfMemoryException

This is regardless of 32-bit or 64-bit OSs. That said, it doesn't stop you from using more than that amount in total, just not on one object.

It is a runtime error because if you keep the long (or other initializing value) within range, it will work.

You can initialize arrays with all integral types: sbyte, char, short, int, and long - all compile; the unsigned variants work too.

Community
  • 1
  • 1
Adam Houldsworth
  • 63,413
  • 11
  • 150
  • 187
4

There is a solution in .Net 4.5-4.6 for allowing large size for an array.

<runtime>
    <gcAllowVeryLargeObjects enabled="true" />
</runtime>

See documentation.

Uwe Keim
  • 39,551
  • 56
  • 175
  • 291
Ardahan Kisbet
  • 641
  • 1
  • 13
  • 33
3

The long is an integral type, so it can be used to define the array; the exception is from building too large of an array, not specifically from using a long.

Ex, this works just fine:

long size = 20;
byte[] array = new byte[size];
Dave Cousineau
  • 12,154
  • 8
  • 64
  • 80