Converting arrays ushort[] to int[] using Buffer.BlockCopy

Question

I'm trying to convert array type of ushort[4k*4k] of values 0-65k to similiar array type of int[] of same values.

It seems to mee that Buffer.BlockCopy is the fastest way to do that. I'm trying the following code:

   ushort[] uPixels = MakeRandomShort(0, 65000, 4000 * 4000);// creates ushort[] array
   int[] iPixels = new int[4000 * 4000];

   int size = sizeof(ushort);
   int length = uPixels.Length * size;
   System.Buffer.BlockCopy(uPixels, 0, iPixels, 0, length);

But iPixels stores some strange values in very strange range +-1411814783, +- 2078052064, etc.

What is wrong, and what I need to do to make it work properly?

thanks!

Have you seen [this](https://stackoverflow.com/a/61886989/1911064) related post? It should answer your question. — Axel Kemper, Nov 06 '22 at 13:56
Blockcopy maybe fast but it can't do the right thing. For an int every value starts at a 4 Byte offset in an array. But short is just a 2 Byte datadata. So essentially you are taking two shorts and throw them in one integer and having half of the iPixel array unchanged. — Ralf, Nov 06 '22 at 14:25
What about linq? int[] output = uPixels.Select(x => (int)x).ToArray(); — jdweng, Nov 06 '22 at 14:53
@AxelKemper I need BlockCopy. Array.ConvertAll is to slow for me. — Евгений Федоров, Nov 06 '22 at 15:48
@jdweng this is even slower than Array.ConvertAll. I need block copy, MarshalCopy or smth similar — Евгений Федоров, Nov 06 '22 at 15:50
@Ralf maybe it can be done with proper offsets, lenght, or smth? Or Convert ushort[] to byte[] ? I need anything that would be faster then Array.ConvertAll or loop over array (iPixels[i] = uPixels[i]) — Евгений Федоров, Nov 06 '22 at 15:52
@ЕвгенийФедоров No the memory layout is different. So a block operation that does not change that will always return the wrong result. You need to change the memory layout and that means copying in small amount (the size of a short as every short needs to be replaced to be correctly interpreted as int). And that means you are at something slower. Presumably as slow as just calling Array.ConvertAll. — Ralf, Nov 06 '22 at 20:25
If you need to optimize here you may looking at the wrong stuff and need to rethink. Do you need it as short in the first place? Do you need to change the complete array? Do you need the complete thing in memory? — Ralf, Nov 06 '22 at 20:27

Axel Kemper · Accepted Answer · 2022-11-07T19:18:42.093

There is a related discussion on GitHub.

To copy an ushort[] to an int[] array does not work with a routine tuned for contiguous memory ranges.

Basically, you have to clear the upper halves of the target int cells. Then, some sort of (parallelized?) loop is needed to copy the actual data.

It could be possible to use unsafe code with pointers advanced in steps of two bytes. The implementation of Buffer.BlockCopy is not visible in the Microsoft source repository. It might make sense to hunt for the source and modify it.

Update

I implemented two C++ functions and did a rough measurement of the resulting performance compared to the C# loop copy.

C# implementation

const int LEN = 4000 * 4000;
for (int i = 0; i < LEN; i++)
{
    iPixels[i] = uPixels[i];
}

C++ implementation SpeedCopy1

// Copy loop with casting from unsigned short to int
__declspec(dllexport) void SpeedCopy1(unsigned short *uArray, int * iArray, int len)
{
    for (int i = 0; i < len; i++)
    {
        *iArray++ = *uArray++;
    }
}

C++ implementation SpeedCopy2

/// Copy loop with unsigned shorts
/// Clear upper half of int array elements in advance
__declspec(dllexport) void SpeedCopy2(unsigned short* uArray, int* iArray, int len)
{
    unsigned short* up = (unsigned short*)iArray;

    memset(iArray, 0, sizeof(int) * len);

    for (int i = 0; i < len; i++)
    {
        *up = *uArray++;
        up += 2;
    }
}

Resulting times:

C# loop copy  27 ms
SpeedCopy1     9 ms
SpeedCopy2    18 ms

Compared to the C# loop, the external C++ function can reduce the copy time down a third.
It remains to be shown, what effect could be gained by multi-threading.

[Source](https://source.dot.net/#System.Private.CoreLib/src/libraries/System.Private.CoreLib/src/System/Buffer.cs,1fbec67d6ee92203,references). But shoudn't help. Its written around copying something in one block (hence the name of the method). There is no copying in small chunks that could be rewritten to reshuffle source and target adresses to get something fitting. And if it would be about copying in small chunks then and were back to something slower. Presumably as slow as the other methods he don't want. — Ralf, Nov 06 '22 at 20:20
Yes, it boils down to a call of `__Memmove()` which in turn is meant for contiguous memory. It is amazing, how convoluted these library functions are implemented. — Axel Kemper, Nov 06 '22 at 20:41

Converting arrays ushort[] to int[] using Buffer.BlockCopy

1 Answers1