0

I'm a little bit out of ideas. With the following code I try to instace a byte array > than 2GB:

var b = Array.CreateInstance(typeof(byte), uint.MaxValue);

Every time it will cause an System.ArgumentOutOfRangeException excpetion with the message that arrays larger then 2GB are not supported.

My App.config is currently the following:

<?xml version="1.0" encoding="utf-8" ?>
   <configuration>
      <startup> 
         <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2"/>
      </startup>
      <runtime>
         <gcAllowVeryLargeObjects enabled="true" />
      </runtime>
   </configuration>

Further the target platform of the project is x64

I'd be grateful for any ideas. Should there be any information missing I will update the question so as soon as possible.

Update 1

I also tried uint.MaxValue

MarvinMcFly
  • 21
  • 4
  • 1
    An array is indexed with an [int] expression. You can't index that array, the maximum value of that expression is Int32.MaxValue. – Hans Passant Jul 20 '19 at 11:14

2 Answers2

0

Just for sanity check, you are trying to allocate 9.223 EB (exabytes) sequential block of memory and that is 9.223×10^9 GB (gigabytes). Simply, but you cannot even do that on x64 machine as some memory is used anyways and that would be maximum.

Instead try to use dynamically growing list:

var b = new List<byte>();

edit:

The maximum index in any single dimension is 2,147,483,591 (0x7FFFFFC7) for byte arrays and arrays of single-byte structures, and 2,146,435,071 (0X7FEFFFFF) for other types.' - source: https://learn.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/gcallowverylargeobjects-element

The effect of gcallowverylargeobjects-element is that you can define multi-dimentsional arrays that exceed 2Gb and in case of other data types you can allocate 2146435071*data_type_size memory. For example int32 consists for 4 bytes, so it would be 8.586 GB (gigabytes).

Community
  • 1
  • 1
Margus
  • 19,694
  • 14
  • 55
  • 103
  • Thank you for your reply! You're completely right, it's a not-so-sense approach. I've reworked the question. uint.MaxValue causes the same exception and would, in my opinion, instantiate a 4GB array. – MarvinMcFly Jul 20 '19 at 11:15
  • The problem with 4GB is about 2x too much, 'The maximum index in any single dimension is 2,147,483,591 (0x7FFFFFC7) for byte arrays and arrays of single-byte structures, and 2,146,435,071 (0X7FEFFFFF) for other types.' - source: https://learn.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/gcallowverylargeobjects-element – Margus Jul 20 '19 at 12:25
  • Based on your quote: What then is the purpose of gcAllowVeryLargeObjects? If no object other than arrays is affected, but the index is limited to 2,147,483,591 (0x7FFFFFC7). – MarvinMcFly Jul 20 '19 at 12:40
  • @MarvinMcFly You can define multi-dimentsional arrays that exceed 2Gb. – Margus Jul 20 '19 at 13:05
  • @MarvinMcFly The purpose is that arrays can be larger than 2GB in _size_. E.g. if you have an array of ints: you can now let it have 2bn elements each 4 bytes long and thus the array being 8GB in size. – ckuri Jul 20 '19 at 13:20
  • Might want to update it as OP has changed from `ulong.MaxValue` -> `uint.MaxValue` – John Jul 20 '19 at 14:04
0

Single-dimensional arrays can't contain more than int.MaxValue elements, even though they can be bigger than 2gb with <gcAllowVeryLargeObjects (e.g new int[int.MaxValue / 2] is ~4gb). To get around this, you must either create a 2 dimensional array, or use a different type, such as

public struct BytePair
{
    public byte First, Second;
}

and then create a BytePair[] of half the size the equivalent byte[] would be

John
  • 598
  • 2
  • 7
  • 22
  • Or he could just use a ValueTuple, e.g. `new (byte First, byte Second)[2_100_000_000];`. – ckuri Jul 20 '19 at 17:08