2

Let B1 and B2 be a dynamic-size storage classes.
(e.g. B1~std::vector<char> B2~std::vector<float>)

In C++11, if I code B1 and B2's move and copy function (rule of five), a class C that contains them as fields will copy/move correctly by default automatically.

class C{
    B1 b1; B2 b2;
};

It works very good.

Problem

Today, I got the profile result + did some test about performance issue.
Main objective: I have to make b1 and b2 of the same instance of C allocate memory near each other :-

b1[0]  b1[1] ... b1[b1.size-1] (minimum gap) b2[0]  b2[1] ... b2[b2.size-1]

If I can, I will get performance boost by 10-20% for the whole program.

My poor solution

I can use a custom allocator like this (pseudo-code):-

class C{
    B1 b1; 
    B2 b2;
    Allocator* allo_; // can be heap allocator 
    public: void reserve(int size){
        //old : b1.reserve(size); b2.reserve(size);  .... so easy
        //new :-
        B1 b1Next; B2 b2Next;
        int nb1=b1Next.howMuchIWant(size); 
        int nb2=b2Next.howMuchIWant(size);
        //^ request amount of bytes needed if capacity="size"
        void* vPtr=allo_->allocate(nb1+nb2);
        b1Next.setMemory(vPtr);
        b2Next.setMemory(vPtr + nb1);  //add "vPtr" by "nb1" bytes
        b1Next=b1;   //copy assignment (not move memory)
        b2Next=b2;   //copy assignment (not move memory)
        b1=std::move(b1Next);   //move memory
        b2=std::move(b2Next);   //move memory 
        //clean up previous "vPtr" (not shown)
    }
};

It works, but the code become far harder to debug/maintain. Not to mention C's move and copy.

In the old version, all copy/move mess appear only in B1 and B2.
Now, the mess appears in every class that use data-structure like B1 and B2 directly.

Question

What are C++ technique/design-pattern/idiom that can help?
To answer, no runable code is required. Pseudo code or just a concept is enough.

I am so regret for not providing MCVE.
Custom allocator and array management are things that really hard for the minimizing.

cppBeginner
  • 1,114
  • 9
  • 27
  • Is it necessary to have all `char` and `int` inside `vector`s? Otherwise you could allocate a single `vector` of bytes (`uint8_t`) and layout the data inside yourself. Data is then always organized in a single memory region and copy and move operations should be easy to implement. – nh_ Sep 11 '17 at 08:28
  • @nh_ It is easy if I have to implement just a single `C`. In real case, there are currently 5 classes like this, and it is increasing. I wish to make the array itself more reusable. Hmm.... – cppBeginner Sep 11 '17 at 08:32
  • Does `B1` really dynamic ?or just set as construction ? as a push_back in `B1` might invalidate `B2`. – Jarod42 Sep 11 '17 at 09:01
  • @Jarod42 Yes, push_back can invalidate everything. `reserve()` is an example - if `b1`'s capacity is not enough, `b2` will also be re-allocated as a side-effect. In practice, `C` is usually queried in read-mode many times, then write-mode many times, .... . – cppBeginner Sep 11 '17 at 09:03

1 Answers1

1

One possibility to improve data locality is going from a struct of vectors to a vector of structs. Instead of 

struct S
{
    std::vector<char> c;
    std::vector<int> i;
};
S data;

use a

struct S
{
    char c;
    int i;
};
std::vector<S> data;

That way, data is always stored together and you don't need to tinker around with custom allocators. Whether this is applicable in your situation primarily depends on two conditions:

  • Is it necessary to have all char (or int) contiguous? E.g. because an API is called regularly that requires a vector of respective type.
  • Is the number of stored char and int equal (at least nearly equal)?
nh_
  • 2,211
  • 14
  • 24
  • In my case, I can't use it because of the reason you already mentioned. (Their size are not equal, `i` have to be contiguous, data locality & cache miss issue, etc.) But thank! It is a good idea if I become desperate. – cppBeginner Sep 11 '17 at 08:27
  • @cppBeginner Are the number of `int` and `char` changing dramatically during execution? Are there any estimated boundaries for `c.size()` and `i.size()`? And, is the order of both `i` and `c` important? – nh_ Sep 11 '17 at 08:30
  • It is hard to answer. I have many custom data structures that have this symptom, e.g. [`SparseMap`](https://programmingpraxis.com/2012/03/09/sparse-sets/), `SetOfInt`, `SimpleArray`. They are so different from each other. They have a common function `reserve(int size)`, and also provide automatic growing 1->2->4->8 if need. Order of `i`/`c` inside its own array is usually important, but order between array `i` and array `c` is not important. – cppBeginner Sep 11 '17 at 08:38
  • @cppBeginner I am afraid that if you need optimal performance, the solution will be somewhat tailored to the specific problem and not generally applicable. I was thinking about a contiguous byte array, which is split in half such that `i`s grow from front to mid while `c`s grow from mid to end. But it is hard to say whether this is beneficial, because reallocation might be necessary quite often, if numbers change regularly. – nh_ Sep 11 '17 at 08:47
  • I currently do like this. If I start at `[c0,c1,i0,i1]`, then I add `c2`, it will become `[c0,c1,c2,-,i0,i1,-,-]`. The performance is good enough. My problem is mainly about maintainability/re-usability of the code. I already use `char[]` and placement new, and it is very fast. :) – cppBeginner Sep 11 '17 at 08:53
  • @cppBeginner There is probably not much you can do about maintainability, except to wrap all low-level hacks inside the class and provide an easy to use API. I am wondering what your re-usability concerns are? If your two-typed vector class is templated it should be usable with arbitrary types and as such be quite re-usable. – nh_ Sep 11 '17 at 09:03
  • I agree that `B1` and `B2` is reusable, but code to manage them (in `C::reserve()`) is dirty and not quite reusable. I have many `C`-like classes. ...... or do you think `C::reserve()` shown in the question is not ugly? (Am I the only one who think it is ugly?) – cppBeginner Sep 11 '17 at 09:09