Prevent vector items from being moved

Question

this is a learning question for me and hopefully others as well. My problem breaks down to having a pointer pointing to content of a vector. The issue occurs when I erase the first element of the vector. I'm not quite sure what I was expecting, I somehow assumed that, when removing items, the vector would not start moving objects in memory.

The question I have is: is there a way to keep the objects in place in memory? For example changing the underlying container of vector? With my particular example, I will remove the pointer access and just use and id for the object since the class needs a ID anyway.

here is a simplified example:

#include <iostream>
#include <vector>

class A
{
public:
    A(unsigned int id) : id(id) {};
    unsigned int id;
};

int main()
{
    std::vector<A> aList;

    aList.push_back(A(1));
    aList.push_back(A(2));

    A * ptr1 = &aList[0];
    A * ptr2 = &aList[1];

    aList.erase(aList.begin());

    std::cout << "Pointer 1 points to \t" << ptr1 << " with content " << ptr1->id << std::endl;
    std::cout << "Pointer 2 points to \t" << ptr2 << " with content " << ptr2->id << std::endl;
    std::cout << "Element 1 is stored at \t" << &aList[0] << " with content " << aList[0].id << std::endl;

}

What I get is:

Pointer 1 points to     0xf69320 with content 2
Pointer 2 points to     0xf69324 with content 2
Element 1 is stored at  0xf69320 with content 2

Deleting items from a vector (and many other operations) will potentially move things around in memory. If you want to avoid this, use node-based containers such as std::list or std::deque. — , Feb 16 '17 at 17:53
a vectors elements are by definition stored in contiguous memory. If you want a container that keeps an element in it place but remebers that it is removed from the container, then this is not a vector — 463035818_is_not_an_ai, Feb 16 '17 at 17:53
if you really want a vector like that, then use std::vector::reserve(), it reserve a space so when you add a new one it will put the element to that reserve space so no reallocation will happen. vector only reallocates when it needs to increase in size — Lorence Hernandez, Feb 16 '17 at 17:56
@LorenceHernandez That doesn't stop `erase` from moving elements forward to fill ion the gap. — NathanOliver, Feb 16 '17 at 17:59
@LorenceHernandez -- Call me silly, but I would not use `reserve()` to defeat the purpose of a vector. Your application must now always check that you're not over the capacity before inserting a new item in the vector. — PaulMcKenzie, Feb 16 '17 at 18:00
Even if it's a learning question, I'd ask you to provide an application example. A regular vector, storing regular data, I'd consider an erase operation to be something that deletes content forever. That is pretty much the idea of it. Can you provide an example of what you want there to be? Also, would it solve your problem if you went with a vector over pointers to A, with the actual content being stored at some other place? Which is also responsible for deleting the content? Because that responsibility clearly needs to be located somewhere. — Aziuth, Feb 16 '17 at 18:02
@Aziuth Basically, by erasing the first object, pointers to the second (and so on) objects are invalid. Objects in a different vector store pointers to my original vector to define relationships between them. The approach (storing pointers instead of objects) would help but my initial goal was to not be required to do that. :'D — ruhig brauner, Feb 16 '17 at 18:08
Perhaps, instead of actually deleting values while you need them to remain in place, you can use a "deleted" or "invalid" placeholder value until you're able to move them? This would allow you to effectively delete values without changing the memory layout, but would add the slight overhead of validating values before use. — Justin Time - Reinstate Monica, Feb 16 '17 at 18:57
@NathanOliver oh sorry, i didnt read the whole question. i thought he was talking about adding element. — Lorence Hernandez, Feb 18 '17 at 13:44

score 1 · Answer 1 · answered Feb 16 '17 at 18:04

1

While you can't achieve what you want exactly, there are two easy alternatives. The first is to use std::vector<std::unique_ptr<T>> instead of std::vector<T>. The actual instance of each object will not be moved when the vector resizes. This implies changing any use of &aList[i] to aList[i].get() and aList[i].id to aList[i]->id.

#include <iostream>
#include <memory>
#include <vector>

class A
{
public:
    A(unsigned int id) : id(id) {};
    unsigned int id;
};

int main()
{
    std::vector<std::unique_ptr<A>> aList;

    aList.push_back(std::make_unique<A>(1));
    aList.push_back(std::make_unique<A>(2));

    A * ptr1 = aList[0].get();
    A * ptr2 = aList[1].get();

    aList.erase(aList.begin());

    // This output is undefined behavior, ptr1 points to a deleted object
    //std::cout << "Pointer 1 points to \t" << ptr1 << " with content " << ptr1->id << std::endl;
    std::cout << "Pointer 2 points to \t" << ptr2 << " with content " << ptr2->id << std::endl;
    std::cout << "Element 1 is stored at \t" << aList[0].get() << " with content " << aList[0]->id << std::endl;

}

Note that ptr1 will point to a deleted object, as such it's still undefined behavior to deference it.

Another solution might be to use a different container that does not invalidate references and pointers. std::list never invalidates a node unless it's specifically erased. However, random access is not supported, so your example can't be directly modified to use std::list. You would have to iterate through the list to obtain your pointers.

answered Feb 16 '17 at 18:04

François Andrieux

28,148
6
56
87

That would help but I can't do that in my use case. Have to prevent dynamic allocations. :( – ruhig brauner Feb 16 '17 at 18:10
`std::array` or a `const std::vector` might be your best bet then – François Andrieux Feb 16 '17 at 18:11
About the list approach: can a list reserve space? – ruhig brauner Feb 16 '17 at 18:11
@ruhigbrauner No, [`list` has no `reserve` method](http://en.cppreference.com/w/cpp/container/list). – François Andrieux Feb 16 '17 at 18:12
@ruhigbrauner `std::vector` implicitly assumes a capability of resizing. It's hard to "turn off". Can you be more specific as to why these solutions aren't acceptable for your situation? – François Andrieux Feb 16 '17 at 18:13
I'm in an audio thread and dynamic allocation of any sort is to expensive and might (not sure this is still the case) result in more serious issues. I have to keep track of events (in this case, key's being held down) so I need to store them. However, they are not erased in the order they came in, so erasing the first event while the second one is still active is a situation that can occur. I have a different set of objects that take "care" of one event each, optionally. So I can't really store them in the objects. .... – ruhig brauner Feb 16 '17 at 18:18
What I'd like to have idealy is a container that: - keeps objects at an address once added to the container - can reserve space - optionally supports random acces But right now I think using raw pointers is a bad design in this case. – ruhig brauner Feb 16 '17 at 18:19
2

@ruhigbrauner These are some very specific requirements. Consider using a `list` with a [custom allocator](http://en.cppreference.com/w/cpp/concept/Allocator) or a `vector` of `unique_ptr` and overload the [operator new](http://en.cppreference.com/w/cpp/memory/new/operator_new) for your type to allocate from a pool. In any case, you will likely want to manage the memory yourself and avoid dynamic allocations. I don't know of any standard container that will respect your requirements without either custom allocators or overwriting operator new. – François Andrieux Feb 16 '17 at 18:21
Thanks for your help anyway. These are some interesting alternatives. :D – ruhig brauner Feb 16 '17 at 18:24

Aziuth · Answer 2 · 2017-02-17T16:11:39.463

Not sure if this is what you want, but how about this:

(only basic layout, you need to fill in the details, also: haven't tested the design, might have some flaws)

template <class T>
class MagicVector {

    class MagicPointer {

        friend class MagicVector;

        private:

        MagicVector* parent;
        unsigned int position;
        bool valid;

        MagicPointer(MagicVector* par, const unsigned int pos); //yes, private!

        public:

        ~MagicPointer();

        T& content();
        void handle_erase(const unsigned int erase_position);
    }

    friend class MagicPointer;

    private:

    vector<T> data;
    vector<std::shared_ptr<MagicPointer> > associated_pointers;

    public:

    (all the methods you need from vector)
    void erase(const unsigned int position);

    std::shared_ptr<MagicPointer> create_pointer(const unsigned int position);

}

template <class T>
void MagicVector<T>::erase(const unsigned int position){
    data.erase(position);
    for(unsigned int i=0; i<associated_pointers.size(); i++){
        associated_pointers[i].handle_erase(position);
    }
}

template <class T>
std::shared_ptr<MagicPointer> MagicVector<T>::create_pointer(const unsigned int position){

    associated_pointers.push_back(std::shared_ptr<MagicPointer>(new MagicPointer(this, position)));
    return std::shared_ptr<MagicPointer>(associated_pointers.back());
}

template <class T>
MagicVector<T>::MagicPointer(MagicVector* par, const unsigned int pos){
    parent = par;
    position = pos;
    if (position < parent->data.size()){
        valid = true;
    }else{
        valid = false;
    }
}

template <class T>
T&  MagicVector<T>::MagicPointer::content(){
    if(not valid){
        (handle this somehow)
    }
    return parent->data[position];
}

template <class T>
void  MagicVector<T>::MagicPointer::handle_erase(const unsigned int erase_position){
    if (erase_position < position){
        position--;
    }
    else if (erase_position == position){
        valid = false;
    }
}

template <class T>
MagicVector<T>::MagicPointer::~MagicPointer(){
    for(unsigned int i=0; i<parent->associated_pointers.size(); i++){
        if(parent->associated_pointers[i] == this){
            parent->associated_pointers.erase(i);
            i=parent->associated_pointers.size();
        }
    }
}

Basic idea: You have your own classes for vectors and pointers, with pointer storing a position in the vector. The vector knows it's pointers and handles them accordingly whenever something is erased.

I'm not completely satisfied myself, that shared_ptr over MagicPointer looks ugly, but not sure how to simplify this. Maybe we need to work with three classes, MagicVector, MagicPointerCore which stores the parent and position and MagicPointer : public shared_ptr < MagicPointerCore>, with MagicVector having vector < MagicPointerCore> associated_pointers.

Note that the destructor of MagicVector has to set all of it's associated pointers to invalid, since a MagicPointer can outlive the scope of it's parent.

πάντα ῥεῖ · Answer 3 · 2017-02-16T18:02:12.847

-1

I was expecting, I somehow assumed that, when removing items, the vector would not start moving objects in memory.

How so? What else did you expect?? A std::vector guarantees a contiguous series of it's contained elements in memory. So if something is removed, the other elements need to be replaced in that contiguous memory.

edited Feb 16 '17 at 18:02

answered Feb 16 '17 at 17:54

πάντα ῥεῖ

1
13
116
190

That just came to my mind as well. I don't know where that came from. Is there an STL object that instead organizes a list around objects that keep their address? – ruhig brauner Feb 16 '17 at 17:55
@Incomputable Generally `std::deque` would not be good. You need a node based container like `std::list`. – NathanOliver Feb 16 '17 at 17:58
@ruhigbrauner if you want to stay with vectors you could store the elements in a vector, but access them through a second vector that holds pointers to the first vector. You could erase elements from the second vector while keeping the first one immutable. Maybe not the best solution, but just came to my mind – 463035818_is_not_an_ai Feb 16 '17 at 17:58
@Nathan std:deque is node based - the nodes happen to contain arrays. – Feb 16 '17 at 18:00
@NathanOliver, from the C++ standard point of view, yes, `std::deque` is not suitable for that. But from what I remember they use a list of arrays, so it might get better cache coherence. – Incomputable Feb 16 '17 at 18:00
3

@NeilButterworth Which means if you call erase in the middle all references and iterators are invalidated. You need a true every element is a node container if you want it to be guaranteed to not move anything. – NathanOliver Feb 16 '17 at 18:01
1

@NathanOliver, in the example he removes from the beginning. So `std::deque` is good for the example. But yeah, erasing in the middle is a problem. – Incomputable Feb 16 '17 at 18:02

Prevent vector items from being moved

3 Answers3