vb.net - Inserting 1-dimensional arrays into a 2-dimensional array

Question

I'm working on an optimization algorithm and need to store some of the data (generated by the algorithm) in a 2-dimensional array called matrix, where row(i) contains the fitness score and parameter values of optimization vector(i).

Dim matrix(vectorCount() - 1, parameterCount()) As Double
Dim params(parameterCount() - 1) As Double

For i As Integer = 0 To vectorCount() - 1
    matrix(i, 0) = vectorScore(i)
    params = vectorValues(i)
    For j As Integer = 0 To params.Length - 1
        matrix(i, j+1) = params(j)
    Next
Next

int vectorCount() returns the number of vectors.
int parameterCount() returns the number of parameters in each vector.
double vectorScore(int vectorIndex) returns the fitness score of a specified vector.
double[] vectorValues(int vectorIndex) returns the parameter values of a specified vector.

My question:
Is there a faster (i.e. more efficient) way to insert params into matrix?

score 4 · Accepted Answer · edited May 23 '17 at 12:19

4

If you want efficiency, relying on "plain" arrays is undoubtedly the best option. The reason why they are so efficient with respect to other alternatives (Collections, List, etc.) is because they contain the minimum amount of information required. If you want fancy functions allowing to sort the information easily/quickly or write complex queries to retrieve data, you shouldn't rely on arrays.

I always rely on a code like the one you wrote and haven't ever had a speed problem (this is really fast). I have done a quick research to make sure that there is no other option, but couldn't find any. The closest thing you have is Array.Copy, although it works just when the arrays have the same dimensions (I, personally, use it just with 1D arrays). In any case, I found an interesting link about Array.Copy() vs. loop performance for 2D arrays (it is in C# but everything is applicable to VB.NET).

Summary: your code is really fast and does not need to be improved. In case of having a valid alternative (Array.Copy working for 2D and 1D, what does not exist), the resulting performance would be just a bit better (and only for small array sizes).

edited May 23 '17 at 12:19

Community

1
1

answered Jul 30 '13 at 11:42

varocarbas

12,354
4
26
37

+1 great answer! In my case loop `ì``could easily have up to 10-20 million iterations and loop ``j``up to 12-15 iterations, which makes a total of at least 120 millions iterations. Is my code even with these numbers a good choice? – bluebox Jul 30 '13 at 12:07
@bluebox thanks. I guess that your answer is also worthy to be known (people too worry about easy coding and inefficient storage alternatives lately) -> +1 to it right away. – varocarbas Jul 30 '13 at 12:14
1

@bluebox actually with these numbers, your code is the only acceptable solution (even in the irrealistic case of having a working Array.Copy). As you can see in the provided link, the bigger the array size (= the bigger the number of iterations), the better is the loop performance. – varocarbas Jul 30 '13 at 12:23
@bluebox I guess that you know that creating too big arrays is not a good idea (at all); so I understand that you are accounting for different arrays or performing the iterations at different moments, but that you are not putting all these records in the same array. – varocarbas Jul 30 '13 at 12:26
Thanks for pointing that out - to be honest I actually didn't think about this issue. It's the first time that I have to deal with huge amounts of data like these.. – bluebox Jul 30 '13 at 12:37
1

@bluebox no problem. Try to use 1D arrays as much as possible and when moving to 2D intend to maintain the first dimension below 1000. It would depend upon the number of arrays and the memory of the given computer; this is an in-the-safest side recommendation. You are the one who has to do performance tests and draw the actual limits under the expected conditions. But this is a completely different story; my answer for your question is clear: is this the best way? Yes. :) – varocarbas Jul 30 '13 at 12:42

score 1 · Answer 2 · answered Jul 30 '13 at 12:52

If you want to use more than one thread, you can use a parrallel for loop.

http://msdn.microsoft.com/en-us/library/dd460713.aspx

Dim matrix(vectorCount() - 1, parameterCount()) As Double
Dim params(parameterCount() - 1) As Double

Parallel.For(0, vectorCount() - 1, Sub(i)
                                      matrix(i, 0) = vectorScore(i)
                                      params = vectorValues(i)
                                      For j As Integer = 0 To params.Length - 1
                                          matrix(i, j+1) = params(j)
                                      Next
                                   End Sub)

score 1 · Answer 3 · answered Jul 30 '13 at 13:26

It depends a lot on the size of the array, for loop is very efficient but with very large array you can see improvement with array.copy or buffer.blockcopy.

Sub Main()

    Const ARR_SIZE_X As Integer = 9999999
    Const ARR_SIZE_y As Integer = 5
    Const DBL_SIZE As Integer = 8

    Dim watch As New Stopwatch

    Dim a1(ARR_SIZE_X) As Double
    Dim a2(ARR_SIZE_y, ARR_SIZE_X) As Double

    For x = 0 To ARR_SIZE_X
        a1(ARR_SIZE_X) = x
    Next

    watch.Start()

    For t = 0 To 10
        For y = 0 To ARR_SIZE_y
            For x = 0 To ARR_SIZE_X
                a2(y, x) = a1(x)
            Next
        Next
    Next

    watch.Stop()
    Console.WriteLine(watch.ElapsedTicks)

    watch.Reset()

    watch.Start()

    For t = 0 To 10
        For y = 0 To ARR_SIZE_y
            System.Buffer.BlockCopy(a1, 0, a2, (ARR_SIZE_X + 1) * DBL_SIZE * y, DBL_SIZE * ARR_SIZE_X)
        Next
    Next

    watch.Stop()
    Console.WriteLine(watch.ElapsedTicks)

    'For y = 0 To 4
    '    For x = 0 To 4
    '        Console.Write(a2(y, x))
    '    Next

    '    Console.WriteLine()
    'Next

    Console.ReadLine()

End Sub

score 1 · Answer 4 · answered Sep 20 '16 at 10:57

Function OneD2TwoD(ByVal xLen As Integer, ByVal yLen As Integer)
    ' (1) populate 1d array
    Dim TwoD(xLen, yLen) As Integer
    Dim OneD((xLen + 1) * (yLen + 1) - 1) As Integer
    For i As Integer = 0 To OneD.GetUpperBound(0)
        OneD(i) = i
    Next
    PrintValues(OneD, "|")
    Console.WriteLine()
    ' (2) Convert 1d array to 2d array/
    Dim z, Row(yLen) As Integer
    For x As Integer = 0 To xLen
        For y As Integer = 0 To yLen
            z = x * (yLen + 1) + y
            TwoD(x, y) = OneD(z)
            Row(y) = TwoD(x, y)
        Next
        PrintValues(Row, "|")
    Next
    Console.WriteLine()
    ' (3) Convert 2d array to 1d array/
    Erase OneD
    ReDim OneD((xLen + 1) * (yLen + 1) - 1)
    For x As Integer = 0 To xLen
        For y As Integer = 0 To yLen
            z = y + x * (yLen + 1)
            OneD(z) = TwoD(x, y)
        Next
    Next
    PrintValues(OneD, "|")
End Function

Public Sub PrintValues(ByVal myList As IEnumerable, ByVal mySeparator As Char)
    ' source: https://msdn.microsoft.com/en-us/library/system.collections.arraylist.add(v=vs.110).aspx
    Dim obj As [Object]
    For Each obj In myList
        Console.Write("{0}{1}", mySeparator, obj)
    Next obj
    Console.WriteLine()
End Sub 'PrintValues

Example for: OneD2TwoD(0, 10)

(1) Populate 1D array

|0|1|2|3|4|5|6|7|8|9|10

(2) Convert 1D array to 2D array

|0|1|2|3|4|5|6|7|8|9|10

(3) Convert 2D array to 1D array

|0|1|2|3|4|5|6|7|8|9|10

Example for: OneD2TwoD(5, 0)

(1) Populate 1D array

|0|1|2|3|4|5

(2) Convert 1D array to 2D array

|0

|1

|2

|3

|4

|5

(3) Convert 2D array to 1D array

|0|1|2|3|4|5

Example for: OneD2TwoD(2, 5)

(1) Populate 1D array

|0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17

(2) Convert 1D array to 2D array

| 0| 1| 2| 3| 4| 5

| 6| 7| 8| 9|10|11

|12|13|14|15|16|17

(3) Convert 2D array to 1D array

|0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17

Thank you for your attention

Please try to avoid just dumping code as an answer and try to explain what it does and why. Your code might not be obvious for people who do not have the relevant coding experience. — Frits, Sep 20 '16 at 11:25

vb.net - Inserting 1-dimensional arrays into a 2-dimensional array

4 Answers4