Remove duplicate elements from std::vector but starting from the front?

Question

How can I delete duplicate elements from a vector but starting from the front?

So

2 3 4 5 2 5 would become 3 4 2 5

1 5 3 1 would become 5 3 1

I would like the solution to be easy to read, and it would be nice if it had good performance as well.

Do you mean easiest to code, or are you bothered about run-time efficiency? — Adrian Mole, Feb 10 '21 at 21:30
OK. "Easiest to *read*" rather than "easiest to *write*" implies a certain degree of elegance. Thinking... — Adrian Mole, Feb 10 '21 at 21:32
Simply take one of the existing algorithms that do it the usual way, and reverse everything: iterate from the end to the beginning, instead of the beginning to the end, and then remove the left-over bits at the beginning of the vector, instead of the end of the vector, by `erase`()ing the left-over bits. Mission accomplished. — Sam Varshavchik, Feb 10 '21 at 21:33
@Sam or even easier from a readability standpoint, make a call to `std::reverse()`, run the standard remove code, and then call `std::reverse()` again. — scohe001, Feb 10 '21 at 21:34
I wonder if "_easiest to read_" isn't in the eye of the beholder. — Ted Lyngmo, Feb 10 '21 at 21:36
@TedLyngmo It seems you closed as opinion-based. If so, I've edited the question slightly to just be a "how-to", and I think it's ok to be open now. — cigien, Feb 11 '21 at 01:39

Vlad from Moscow · Accepted Answer · 2021-02-10T21:47:55.007

If a container supports bidirectional iterators then it is unimportant from which side of the container you are trying to remove duplicate elements because you can use reverse iterators.

Here is a demonstrative program.

#include <iostream> 
#include <vector> 
#include <iterator> 
#include <algorithm> 
  
template <typename ForwardIterator> 
ForwardIterator remove_duplicates( ForwardIterator first, ForwardIterator last ) 
{ 
    for ( ; first != last; ++first ) 
    { 
        last = std::remove( std::next( first ), last, *first ); 
    } 
      
    return last; 
} 
  
int main() 
{ 
    std::vector<int> v = { 1, 2, 3, 4, 5, 4, 3, 2, 1 }; 
      
    for ( const auto &item : v ) std::cout << item << ' '; 
    std::cout << '\n'; 
  
    v.erase( remove_duplicates( std::begin( v ), std::end( v ) ), std::end( v ) ); 
      
    for ( const auto &item : v ) std::cout << item << ' '; 
    std::cout << '\n'; 
  
    std::cout << '\n'; 
      
    v.assign( { 1, 2, 3, 4, 5, 4, 3, 2, 1 } ); 
  
    for ( const auto &item : v ) std::cout << item << ' '; 
    std::cout << '\n'; 
      
    v.erase( std::begin( v ), remove_duplicates( std::rbegin( v ), std::rend( v ) ).base() ); 
     
    for ( const auto &item : v ) std::cout << item << ' '; 
    std::cout << '\n'; 
}

The program output is

1 2 3 4 5 4 3 2 1 
1 2 3 4 5 

1 2 3 4 5 4 3 2 1 
5 4 3 2 1

eerorika · Answer 2 · 2021-02-10T23:33:14.943

An asymptotically efficient algorithm (N log N):

Create a range of iterators each pointing to the original container
Sort the iterators with a stable sort. Use a comparison function that indirects through the iterator.
Remove consecutive duplicates (std::unique) using reverse iterator and similar custom comparison function.
std::remove_if from the vector with a predicate function that removes elements whose iterator isn't in the secondary container.

It's a bit more complex than the O(N*N) solution. Here is an example implementation. I cannot guarantee that it is correct:

template<class It, class Sentinel>
auto remove_duplicates(It first, Sentinel last)
{
    std::vector<It> iterators;

    auto iterator_generator = [it = first]() mutable {
        return it++;
    };
    std::generate_n(
        std::back_inserter(iterators),
        last - first,
        iterator_generator);

    auto iterator_compare = [](const auto& l, const auto& r) {
        return *l < *r;
    };
    
    std::stable_sort(
        iterators.begin(),
        iterators.end(),
        iterator_compare);

    auto iterator_eq = [](const auto& l, const auto& r) {
        return *l == *r;
    };
    auto last_unique = std::unique(
        iterators.begin(),
        iterators.end(),
        iterator_eq);
    iterators.erase(last_unique, iterators.end());

    auto keep_generator = [it = first]() mutable {
        return it++;
    };
    std::vector<bool> remove(last - first, true);
    for(auto it : iterators) {
        auto index = it - first;
        remove[index] = false;
    }
    
    auto remove_predicate = [index = 0, remove = std::move(remove)](const auto& el) mutable {
        return remove[index++];
    };
    return std::remove_if(first, last, std::move(remove_predicate));
}

// usage with reverse iterators
v.erase(
    v.rend().base(),
    remove_duplicates(v.rbegin(), v.rend()).base());

Remove duplicate elements from std::vector but starting from the front?

2 Answers2