Allocating Jagged Array C#

Question

It seems like there is no good way to allocate a jagged array in C#. It takes several orders of magnitude longer to allocate a jagged array than to allocate a 2D array. Is there any way around this? Is there a better way? I want the allocation of the jagged array to be similar in time the allocation of the 2D array.

Edit: To be clear, every byte array in the jagged array will be length 8. But the 2D array does not work, because I cannot pass it to the API that I need to pass it to. The API I need to use takes an array of byte arrays.

This takes 20 seconds:

public void CreateHashByte()
{
    byte[][] holder = new byte[100000000][];
    for (int k = 0; k < 100000000; k++)
    {
        holder[k] = new byte[8];
    }
}

This is almost instantaneous:

public void CreateHashByte2()
{
    byte[,] holder = new byte[100000000, 8];
}

Edit 2: This is pretty fast, about a half second, but it requires me to rewrite the API to unwrap the byte array. A half second still seems like a long time to just set aside a GB of memory.

public class Hbyte
{
    public byte[] data { get; set; } = new byte[8];
}

public Hbyte[] CreateHashByte3()
{
    Hbyte[] holder = new Hbyte[100000000];
    return holder;
}

What about managing the lengths by yourself and use 1d array instead? I doubt it's any faster. — Luke Vo, May 30 '23 at 03:55
ProgrammingLlama, I need to create an array of byte arrays, which I will then populate and pass to an API that takes an array of byte arrays. — Major Major, May 30 '23 at 05:37
In the 2D array case, you're allocating a single array. In the jagged array case, you're allocating 100000001 arrays some of which will trigger GC. Obviously, this is going to take longer than allocating a single array. — Brian Rasmussen, May 30 '23 at 06:05
Brian Rassmussen: Thanks. Yes, you are restating the issue. I think I am running into a fundamental difference between a C# array, which is an object, and a C array, which is just some space in the stack. Still, 20 seconds seems too long. When you look at it like that, though, it's surprising that the wrapper solution I added is so much faster than the ragged array. — Major Major, May 30 '23 at 06:27

score 1 · Answer 1 · answered May 30 '23 at 04:12

Using 1D array is faster but you need extra space to manage the indices (or else without it the access would take a long time to know where any element is at).

For my test (ms is the unit):

Jagged Array took 8418
1D Array took 397

using System.Diagnostics;

const int TestLength = (int)1E8;
var lengths = new int[TestLength];
for (int i = 0; i < TestLength; i++)
{
    lengths[i] = i % 10;
}

var timer = new Stopwatch();

// Using C# jagged array
timer.Restart();
var arr1 = new byte[TestLength][];
for (int i = 0; i < TestLength; i++)
{
    arr1[i] = new byte[lengths[i]];
}

Console.WriteLine("Jagged Array took " + timer.ElapsedMilliseconds);

// Using 1d array
timer.Restart();
var indices = new long[TestLength]; // You can reuse the lengths array
long currIndex = 0;
for (int i = 0; i < TestLength; i++)
{
    indices[i] = currIndex;
    currIndex += lengths[i];
}

var arr2 = new byte[currIndex];

Console.WriteLine("1D Array took " + timer.ElapsedMilliseconds);

// To use 1d array:
var valueAt21 = arr2[indices[2] + 1];

Luke Vo, thanks for taking a look at this. I don't think it solves my problem, unless I'm misunderstanding your solution. I need to allocate an array of byte arrays. In the end, I need an array of byte arrays of length 8. — Major Major, May 30 '23 at 05:35

Allocating Jagged Array C#

1 Answers1