2

I have a lot of data in a list, say several kbytes in each element, I would like to extract each by each to do some numeric processing. These data are originally stored as float[]. Since the processing involves a lot of indexing and global calculation, I think valarray might be easy to program. But if I use valarray, I may have to copy from the array to the valarray first, and then copy back to the array. Is there any way to avoid this? Any way such that to let me work directly on the arrays? Or do you have better ways to solve similar problems?

Jota
  • 234
  • 2
  • 11
shangping
  • 989
  • 2
  • 9
  • 25
  • 1
    What is the actual problem that you are trying to solve? – Lightness Races in Orbit Jul 27 '11 at 16:26
  • @tomalak: just want to process a lot of data which is internally stored in memory in a list. Each element in the list is a data structure or a class. The processing mainly involves numerical calculations on all or some values in an array inside the data structure. Hope I make it clear this time. – shangping Jul 27 '11 at 16:32
  • I don't understand the issue at all. Perhaps I'm just being thick. – Lightness Races in Orbit Jul 27 '11 at 16:51
  • @Tomalak, I think Shangping has an ordinary array, but wants to use some of the operations that `valarray` provides. To do so would require copying all the data from the ordinary array into the valarray. The question is how to avoid doing that copy. – Rob Kennedy Jul 27 '11 at 17:35
  • Is the cost of the copies significant compared to the cost of the numeric processing you are doing? Have you measured it? – Alan Stokes Jul 27 '11 at 20:33

2 Answers2

1

Warning: ugly hack.

On my system (MS Visual Studio) the valarray class is defined like this:

template<class _Ty>
class valarray
{
    ...
private:
    _Ty *_Myptr;    // current storage reserved for array
    size_t _Mysize; // current length of sequence
    size_t _Myres;  // length of array
};

So i can build my own class that has the same layout (with a good level of confidence):

struct my_valarray_hack
{
    void *_Myptr;
    size_t num_of_elements;
    size_t size_of_buffer;
};

Then create an empty valarray and overwrite its internal variables so it points to your data.

void do_stuff(float my_data[], size_t size)
{
    valarray<float> my_valarray;
    my_valarray_hack hack = {my_data, size, size};
    my_valarray_hack cleanup;

    assert(sizeof(my_valarray) == sizeof(hack));

    // Save the contents of the object that we are fiddling with
    memcpy(&cleanup, &my_valarray, sizeof(cleanup));

    // Overwrite the object so it points to our array
    memcpy(&my_valarray, &hack, sizeof(hack));

    // Do calculations
    ...

    // Do cleanup (otherwise, it will crash)
    memcpy(&my_valarray, &cleanup, sizeof(cleanup));
    // Destructor is silently invoked here
}

This is not a recommended way of doing things; you should consider it only if you have no other way to implement what you want (maybe not even then). Possible reasons why it could fail:

  • Layout of valarray may be different in another mode of compilation (examples of modes: debug/release; different platforms; different versions of Standard Library)
  • If your calculations resize the valarray in any manner, it will try to reallocate your buffer and crash
  • If the implementation of valarray assumes its buffer has e.g. 16-byte alignment, it may crash, do wrong calculations or just work slowly (depending on your platform)
  • (I am sure there are some more reasons for it not to work)

Anyway, it's described as "undefined behavior" by the Standard, so strictly speaking anything may happen if you use this solution.

anatolyg
  • 26,506
  • 9
  • 60
  • 134
1

The valarray type does not provide any way to use an existing array for its data store; it always makes a copy for itself. Instead of storing your data in an ordinary array, store the values directly in the valarray from the start. Call v.resize to set the size, and either assign values into it with the [] operator, or use &v[0] to get a pointer to the first value and use it as you would an iterator or buffer pointer — elements of a valarray are stored contiguously in memory.

Rob Kennedy
  • 161,384
  • 21
  • 275
  • 467
  • Thanks, I understand what you say. The problem is I cannot change the previous array design which involves too many changes. – shangping Jul 27 '11 at 17:35