0

I'm working with a loop with a very high iteration count, and so code within it will be quite performance critical. At some point in the loop I take a previously populated array of doubles and pass it to a function that for our purposes we can consider just a sum function, that being it needs to loop over each element in the array and combine it into a value.

Then later I need to take this first array and pass it again to the same function but with the last element changed. The immediate approach that comes to mind is to copy the first n-1 elements into a new buffer, then set this last element as needed. However, I feel as this is in a large loop this would be inefficient, so I've been thinking of ways to minimize this overhead.

In particular I'm wondering if it would be possible to use mmap to map the first n-1 elements of our second buffer onto the first n-1 elements of the original buffer, and then change the last, unmapped element as needed. This should be able to work using copy-on-write, so though a second buffer is allocated there isn't actually any data copying happening (obviously as long as the second buffer is only read). I could then be able to pass this second buffer to the same function which will act on it transparently without any knowledge of the mapping, the addresses up to n-1 translating into the first buffer, and the nth address actually being in the second buffer.

I've tried to throw together some code to see if this can be done and work as expected, but my main problem right now is mmap only maps arrays to file descriptors, rather than arrays to arrays.

My questions then are

  1. Can I achieve this behavior using mmap?
  2. How can I implement this without using file descriptors, and if I have to use them, how best should I do it?

I will mention as well that it would be best if the solution remains completely transparent to the function these buffers are passed to, due to how this code base is intended to be used.

muke
  • 306
  • 2
  • 11
  • 1
    "The immediate approach that comes to mind is to copy the first n-1 elements into a new buffer". Not really. To me it seems more tempting to change the last element and use the same array again. Please explain why you discarded that option. – Yunnosch May 03 '20 at 18:18
  • Could you not create a container that holds multiple vectors and traverses them continuously by defining correct iterator advancement mechanisms? – user6386155 May 03 '20 at 18:18
  • 1
    Can you calculate the effect of the last element out of the "sum" (with an actual sum that is of course easy)? If yes do so and only calculate the difference effect of the changed last element into the previous result. That would yield much better speed performance than playing with memory. – Yunnosch May 03 '20 at 18:19
  • @Yunnosch I need to save the original buffer for future use, sorry I didn't mention this. And also no it won't be guaranteed I can treat the 'sum' function in this way either. – muke May 03 '20 at 18:24

1 Answers1

1

mmap would not be suitable for this. And, it's not really needed.

When calling the sum function the second time:

  1. save the last element value
  2. set the last element to the modified value
  3. call the sum function
  4. restore the last element value from its saved value

Here's some code that illustrates what I mean:

double
sumall(double *arr,size_t cnt)
{
    double sum = 0;

    for (size_t idx = 0;  idx < cnt;  ++idx)
        sum += arr[idx];

    return sum;
}

double
sumall_newlast(double *arr,size_t cnt,double newval)
{
    double oldval;
    double sum;

    // save original value of last element
    oldval = arr[cnt - 1];

    // set new value for last element
    arr[cnt - 1] = newval;

    sum = sumall(arr,cnt);

    // restore original value of last element
    arr[cnt - 1] = oldval;

    return sum;
}

int
main(void)
{
    double arr[1000];

    sumall(arr,1000);
    sumall_newlast(arr,1000,37.285);

    return 0;
}

UPDATE:

I realize this may not be possible, depending on the actual algorithm, but ...

It seems wasteful to call the sum function a second time, process the entire array, just for a change to the last element. This would be particularly true if dealing with large data arrays.

It would seem much faster to take the result of the first sum call and just apply a correction to result based on the difference of the original last element value and the modified one.

An alternative might be to call the function with the array but shorten the count by 1. Then call it [a third time] with a pointer to a temp array that has the single element of the modified input value. Then, combine the last two return values.

Craig Estey
  • 30,627
  • 4
  • 24
  • 48
  • That is why I asked whether it is possible. I think you prematurly assume things concerning the "quasi sum". You are aware that OPs "sum" is not an actual sum, aren't you? – Yunnosch May 03 '20 at 18:21
  • Saving the overwritten values then restoring them would be a much simpler solution yes, I feel silly for not having realised this now. Thanks. – muke May 03 '20 at 18:26
  • @Yunnosch Yes, of course. What I posted will work, regardless of the complexity of the "sum" function. – Craig Estey May 03 '20 at 18:28