Remove Duplicate Entries in a C++ Vector

Question

Just want to remove duplicates. Pool is vector<pair<string, int>> but I seem to miss some elements at the start of the vector somehow. Can anyone verify the logic of the removal? Thanks :)

Pool Master::eliminateDuplicates(Pool generation)
{
    for(int i = 0; i < generation.size(); i++)
    {
        string current = generation.at(i).first;

        for(int j = i; j < generation.size(); j++)
        {
            if(j == i)
            {
                continue;
            }
            else
            {
                string temp = generation.at(j).first;
                if(current.compare(temp) == 0)
                {
                    Pool::iterator iter = generation.begin() + j;
                    generation.erase(iter);
                }
            }
        }
    }

    return generation;
}

An easier (and likely faster way than the `O(n^2)` way that it currently takes) of doing this is adding all the elements to a `std::set` and then back to a `std::vector`. — Yuushi, May 10 '13 at 06:03
I thought sets would only work based on the integer data type? I'm comparing for string duplicates. EDIT yes, sorry typo ;) — Jarrod Cabalzar, May 10 '13 at 06:05
Is this statement if(j == i){continue;} necessary? You can just start the loop from i+1. — Quazi Marufur Rahman, May 10 '13 at 06:06
@QuaziMarufurRahman Could result in a segmentation fault if i was the last element. — Jarrod Cabalzar, May 10 '13 at 06:07
`std::set` works based upon either `operator<`, or you can give it a comparison function it can use. It'd be a pretty useless set if it only worked on integers! — Yuushi, May 10 '13 at 06:07
In any case, you don't have to use set. You can sort your vector, and use `std::unque`. That would be O(Nlog(N)) — juanchopanza, May 10 '13 at 06:09

score 24 · Answer 1 · answered May 10 '13 at 06:16

24

If you don't mind sorting the vector, then you can use std::unique. That would be O(Nlog(N))

#include <iostream>
#include <algorithm>
#include <vector>

int main() 
{
    std::vector<int> v{1,2,3,1,2,3,3,4,5,4,5,6,7};
    std::sort(v.begin(), v.end()); 
    auto last = std::unique(v.begin(), v.end());
    v.erase(last, v.end());
    for (const auto& i : v)
      std::cout << i << " ";
    std::cout << "\n";
}

answered May 10 '13 at 06:16

juanchopanza

223,364
34
402
480

7

+1 Someone should write a wiki / FAQ entry for all the bread and butter vector usages. – TemplateRex May 10 '13 at 06:32
2

@rhalbersma, SO should maintain a list of the most commonly asked questions on popular topics, like Top 10 C++ Questions or something. That would be handy. :D – Jarrod Cabalzar May 10 '13 at 12:48
I wonder why in all the answers about using `std::unique` nobody seems worth to mention that `unique` does not consider the last element. – tomi.lee.jones Dec 21 '13 at 14:21
@tomi.lee.jones I don't really understand what you mean here. `std::unique`'s behaviour seems quite intuitive to me. – juanchopanza Dec 21 '13 at 14:25
@juan from the documentation, "removes all consecutive duplicate elements from the range [first, last)", so (1,2,3,1) will be (1,2,3,1). How is that intuitive? I know why the range is right-open and why it must be, but unless you are sure there is no dupe on the last position, unique seems useless. – tomi.lee.jones Dec 21 '13 at 15:10
2

@tomi.lee.jones Because all standard library algorithms work on open ranges, and usually we pass begin, end, which is open in that way. – juanchopanza Dec 21 '13 at 15:45
1

If you are going to sort, why not `std::sort`? – May 18 '14 at 14:57

diwatu · Accepted Answer · 2013-05-10T06:19:15.527

4

This is a very common issue.

Because after you erase an element the position j pointed will skip one element due to the j++ on the for loop. the easiest solution to solve the problem based on your code is to add j-- after generation.erase(iter):

  generation.erase(iter);
  j--;

edited May 10 '13 at 06:19

answered May 10 '13 at 06:10

diwatu

5,641
5
38
61

Remove Duplicate Entries in a C++ Vector

2 Answers2

Linked