C++ Deleting from Vector Efficiency Problem

Question

I have 2 versions of the exact same program, maze generator, written in Python and C++. What i thought after optimizing it in Python is that, if i rewrite it in C++ it will be much faster and more efficient. However, i discovered one surprising thing (well not that surprising when you think about it). For Python: my algorithm picks a random item from the list, processes it and deletes it from there afterwards. For C++: all the same, but instead of list i use vector. Deleting elements from a vector in C++ is much slower than doing the same for a list in Python because vector's elements shift when you delete on of them. My question is: What is the best data structure in C++ that can be indexed and performs deletion of its items faster than vector?

Right now C++'s deletion takes 5-6 times more time than Python's on average.

use a linked list (`std::list`), or a vector of pointers on elements. — Jean-François Fabre, Mar 10 '20 at 19:34
Why pick a random element? If you start from the end of the vector, and work your way forward you will have the fastest speed you can get. — NathanOliver, Mar 10 '20 at 19:35
picking a random element is probably to generate a random maze — Jean-François Fabre, Mar 10 '20 at 19:36
that's what my vector contains exactly - pointers. It doesn't help though because they are legit data. I tried using std::list, but it turned out to be even slower. — steak_Overcooked, Mar 10 '20 at 19:37
a C++ vector of pointers is exactly the same thing as a python `list`. — Jean-François Fabre, Mar 10 '20 at 19:38
i pick a random element in order to generate a random maze, sadly i can't pick them in order from the beginning or the end — steak_Overcooked, Mar 10 '20 at 19:38
Then you can [`shuffle`](https://en.cppreference.com/w/cpp/algorithm/random_shuffle) the vector first, then start from the end of the vector, and work your way forward — NathanOliver, Mar 10 '20 at 19:38
Are you measuring the time of optimized C++ build? I have seen plenty of questions asked about performance of C++ vs some other language, when one was using unoptimized C++ builds. Optimizations in C++ aren't enabled by default. — Algirdas Preidžius, Mar 10 '20 at 19:44
I added screenshots of the time measurements just for that one line - deleting an item form a data structure in both languages — steak_Overcooked, Mar 10 '20 at 19:47
@steak_Overcooked We need to see your build settings. All of this information you're showing us means nothing if you are timing an unoptimized, "debug" build. — PaulMcKenzie, Mar 10 '20 at 19:50
@TedLyngmo it looks even uglier as a text, i brought screenshots down though — steak_Overcooked, Mar 10 '20 at 19:55
@steak_Overcooked The thing is not everyone can see pictures. For those people even ugly text is better then something they can't see — NathanOliver, Mar 10 '20 at 19:57
You could also just say that python deletes take less than 0.001ms in average and that the C++-version takes up to 9ms, maybe 5 or 6 in average. Would be interesting to see _how_ you meassure this, _what_ you actually meassure and _how_ you compile it, to sum up the previous comments. Those numbers alone are not really usefull. — Lukas-T, Mar 10 '20 at 20:00
@steak_Overcooked Ugly doesn't matter much. Copy/paste as text (and put it in `code` segments). Perhaps you don't need that much from each output? — Ted Lyngmo, Mar 10 '20 at 20:02
Thanks to @NathanOliver, it helped a lot and now my algorithm got 10-12 times faster than was before and 30% faster that on Python. Appreciate you help! Thank you every one for your suggestions too! :) — steak_Overcooked, Mar 10 '20 at 20:09
@steak_Overcooked No problem. Make sure you are also compiling in release mode (if you are using and IDE) or using `-O2` or `-O3` if you are using the command line to turn on optimizations. If those aren't on then you are also leaving a lot performance on the table. — NathanOliver, Mar 10 '20 at 20:11
@NathanOliver wow, what? is it a trick or something, it is now like 100 times faster. I still need to learn a tone of things about VS... How do i close the thread btw? I can't mark up your comment as the answer — steak_Overcooked, Mar 10 '20 at 20:13
@steak_Overcooked Un like a lot of languages, C++ gives you a lot of control. By default when you compile code it is done in "debug" mode. That means the binary that is generated is a literal translation of the code you used. This is great for debugging, but can be very slow because there is a lot unnecessary work going on. When you put it into release mode, the compiler's optimizer runs and gets rid of all the unneeded code and also does performance increases like using SIMD instructions. This is why it is very important to make sure that is on when comparing speeds to other languages. — NathanOliver, Mar 10 '20 at 20:18
For instance if you recompile your original code where you randomly picked and compile it in release mode, it should be as fast or faster then the python code. — NathanOliver, Mar 10 '20 at 20:20
@steak_Overcooked You don't "close a thread" on stack overflow. You accept an answer that you find most useful. [See here](https://stackoverflow.com/help/someone-answers) — eerorika, Mar 10 '20 at 20:22
@eerorika i know, but i got the answer in the comments, there's no check mark — steak_Overcooked, Mar 10 '20 at 20:24
@NathanOliver How much difference does it make usually? Is their stated "now like 100 times faster" realistic? — Kelly Bundy, Mar 10 '20 at 20:30
@HeapOverflow It can have no effect, it could have 100X effect, or it can be somewhere in between or even more. It all depends on the code and how well it optimizes. I typically see at least a 10X increase in performance when turning on optimizations. Getting a 100X increase is definitely in the realm of possibility. In this case optimizations turned on plus a better algorithm makes it very realistic. — NathanOliver, Mar 10 '20 at 20:35
@NathanOliver It's not C++, it's the compiler. C++ itself gives you no control over the optimizations made. — S.S. Anne, Mar 10 '20 at 20:45

eerorika · Answer 1 · 2020-03-11T15:50:01.790

3

std::list is a linked list and has constant complexity removal of arbitrary elements compared to the linear complexity of a std::vector which is a dynamic array. For small number of elements, vector can still be faster due to better use of cache, but asymptotic complexity determines the speed with large number of elements.

That said, in order to pick a random element from a linked list for removal, you need to use an auxiliary data structure to achieve less than linear complexity.

Also, python list also has linear complexity for deletion, so it is unclear why you would assume it to be faster.

Potentially more efficient for random removal could be a rope data structure that consists of varying sizes of segments. But there is no container implemented with such data structure in the standard library.

More about your specific program than random or arbitrary removals: It seems that better algorithm could potentially be an option:

Use a vector. Move the last valid element over the "removed" and erase the moved from object that is at the end of the vector. This algorithm can be used unless the order of the valid section is important.

edited Mar 11 '20 at 15:50

answered Mar 10 '20 at 19:51

eerorika

232,697
12
197
326

However: Selecting a random element in a `std::vector` is constant time, and selecting a random element in a `std::list` is linear time. I'd expect the `std::vector` to be faster even with the removals, due to cache hits vs cache misses. – Mooing Duck Mar 10 '20 at 19:54
@MooingDuck Depends on how the random element is chosen. It doesn't have to be linear if an auxiliary data structure is used. – eerorika Mar 10 '20 at 19:56
That's true, if you already have a vector of list::iterator to use. But then you have to remove the iterator from _that_ vector. You could have a randomized vector of list::iterators to avoid the vector removal, but now we're a long way from apples to apples, you could just randomize teh original vector to avoid those removals. So in all cases: vector wins. – Mooing Duck Mar 10 '20 at 19:58
Strousup has a talk about this here: https://www.youtube.com/watch?v=YQs6IC-vgmo – Mooing Duck Mar 10 '20 at 19:59
The summary is that for random insertions and removals, the `std::vector` is _much_ faster than the `std::list` for all N. – Mooing Duck Mar 10 '20 at 20:06
@MooingDuck To go along with that: https://baptiste-wicht.com/posts/2012/11/cpp-benchmark-vector-vs-list.html – NathanOliver Mar 10 '20 at 20:09
@MooingDuck With the vector of iterators, which is always used in random, there is no need to maintain the order of elements so erasure can be constant. – eerorika Mar 10 '20 at 20:15
@eerorika: On "python list also has linear complexity for deletion, so it is unclear why you would assume it to be faster": Python `list`s are dynamically sized arrays of raw pointers. The vast majority of the time (the unusual case being when it resizes the underlying storage to free memory), deletion from a Python `list` is equivalent to `memmove` (plus possible object cleanup if the element held the last reference to the object). If they're using value-oriented C++ objects (especially if they're non-POD), the algorithmic cost may be the same, but the overhead could be much higher. – ShadowRanger Mar 11 '20 at 15:47

score 0 · Answer 2 · answered Mar 10 '20 at 20:16

Try creating a specialized vector class that instead of actually deleting an item when you want to delete it, it swaps the item to the end of the vector to a garbage section.

Here's a code example you can use to test performance:

#include <vector>
#include <iostream>
using namespace std;

template <typename T>
struct FastDelVec {
    vector<T> data;
    int deleteIndex;

    FastDelVec(int size) {
        data.resize(size);
        deleteIndex = size - 1;
    }

    void Delete(int index) {
        swap(data[index], data[deleteIndex]);
        --deleteIndex;
    }

    size_t size() {
        return deleteIndex + 1;
    }
};

int main() {
    FastDelVec<int> v(100);
    for (int i = 0; i < 100; ++i) {
        v.data[i] = i;
    }

    v.Delete(30);
    v.Delete(5);
    v.Delete(21);

    cout << v.size() << endl;


    system("pause");
    return 0;
}

There are several other tricks you can try too. This defers deleting until a later time, because constantly freeing heap memory and potentially resizing the vector has a performance hit

Thank you, but i already found the solution by shuffling vector beforehand and working from the end to the beginning :) — steak_Overcooked, Mar 10 '20 at 20:18

C++ Deleting from Vector Efficiency Problem

2 Answers2