2

I'm trying to adapt a vendor's c# example code for interfacing with a PCI-Express device. The code basically allocates a large buffer as an int array, and then pins it via the fixed keyword before handing it off to hardware to be filled with data.

This works great, but it eventually fails because .Net is limited to ~2 billion elements in an array. I can push the limit out to 16 GB by using an array of Long and gcAllowVeryLargeObjects keyword, but eventually I still run into .Net limitations.

In unmanaged code I could call VirtualAlloc and request 40 or 50GB directly, however its not clear to me if this is possible in c# and I haven't been able to find any good example code. I realize I could be doing this in a different language, but on Windows at least I'm more familiar with .Net, and aside from this relatively small portion of the program, there is very little hardware-specific code so I'd like to try and stick with what I have.

user1850479
  • 225
  • 2
  • 12
  • 1
    "~2 billion"? In a 64 bit environment you can create arrays larger than 2 GB and even indices aren't limited to be Int32 (see Array.CreateInstance). – Adriano Repetti Apr 09 '14 at 07:29
  • What's your exact question? You can use `VirtualAlloc` from C#, too. – aevitas Apr 09 '14 at 07:36
  • No even In a 64 Bit CLR app there's a hard limit of 2GB per object! So it's not possible what he want with managed code. – user743414 Apr 09 '14 at 07:54

2 Answers2

8

You can pinvoke VirtualAlloc. The signature is

[DllImport("kernel32.dll", SetLastError=true)]
static extern IntPtr VirtualAlloc(IntPtr lpAddress, UIntPtr dwSize, AllocationType lAllocationType, MemoryProtection flProtect);

You can find most pinvoke structures and signatures on pinvoke.net: VirtualAlloc

Alternatively, take a look at the AllocHGlobal func

jakobbotsch
  • 6,167
  • 4
  • 26
  • 39
1

gcAllowVeryLargeObjects should work just fine, are you sure it doesn't? You may have to explicitly set the target CPU to x64 only.

In any case, you could use a hack struct to get a large value-type (which you can use as an array):

unsafe struct Test
{
  public fixed byte Data[1024];
}

unsafe void Main()
{
    Test[] test = new Test[16 * 1024 * 1024];

    // We've got 16 * 1024 * 1024 * 1024 here.
    fixed (Test* pTest = test)
    {

    }
}

This does have its limits (the unsafe structure has a maximum size), but it should get you where you need to be.

However, it might be better idea to simply call VirtualAlloc through P/Invoke. Or better, use Marshal.AllocHGlobal, it should really be doing the same thing (although you can't specify any parameters except for the size).

Luaan
  • 62,244
  • 7
  • 97
  • 116
  • Thank you, this looks like a very simple solution. It did not occur to me to try an array of struct. I will also look into the pinvoke method above as it is probably a less hackish method. – user1850479 Apr 09 '14 at 18:08