Removing duplicate words from string

Question

I have a string like ABA AAB BAA BAA ABA AAB. I want to remove duplicate words and thus get the output ABA AAB BAA. However, when I run the code below, the output is ABA AAB BAA BAA ABA AAB:

// I'm continuously pushing the string
int S = a.size() 
set <string> s;
for(int i = 0; i < S; i++) {
    s.insert(a);
}
for(auto M : s) {
    cout << M << endl;
}

How to remove duplicate words from a string in C++?

Hi Rohan, Welcome to SO, please edit your question and make it clear. What issue you are facing?.. what is the actual result and what is the expected result. — ssilas777, Jul 22 '21 at 11:22
I'm using c++ whenever I just try to remove the duplicated words which is I mention on top of the title — Rohan Gupta, Jul 22 '21 at 12:02

score 1 · Answer 1 · answered Jul 26 '21 at 13:30

You are on the right track. A std::set is a valid approach.

The problem that you have is that you did not extract single words from the string. You always try to insert the same string into the set. That cannot work.

If you have one string containing "ABA AAB BAA BAA ABA AAB", you want to have substrings "ABA", "AAB", "BAA", "BAA", "ABA", "AAB"

And, insert those into the set. So, you need to split the string into many substrings, so, words. Or, you can also say: "extract" substrings from the string.

Unfortunately the std::string has no extraction operator implemented, because it is not a stream. But there is a standard procedure solving that issue. You can put the std::string into a std::stringstream and then extract from there.

For example, you could write:

#include <iostream>
#include <string>
#include <sstream>
#include <set>

int main() {

    const std::string test{"ABA AAB BAA BAA ABA AAB"};
    std::istringstream iss{test};

    std::set<std::string> uniqueWords{};

    std::string word{};
    while (iss >> word) uniqueWords.insert(word);

    for (const std::string& word : uniqueWords) std::cout << word << '\n';

    return 0;
}

score 0 · Answer 2 · answered Jul 26 '21 at 14:37

It might be helpful for you to understand what you wrote. I'm supposing it doesn't mean what you think it does, but if there's a simple conceptual problem I can't identify it from this small sample.

// I'm continuously pushing the string

You're not pushing anything, anywhere, so I don't understand the comment.

int S = a.size()

This is missing a ; so it won't compile. This gets the number of characters in the string a. You are storing it in a variable of type int, though size() returns a size_t. You did not define it as const but it never changes.

set <string> s;
for(int i = 0; i < S; i++) {
    s.insert(a);
}

You are inserting the string a into the set, over and over again. That doesn't make sense, since a set doesn't store duplicates, and this has nothing to do with the individual words in a.

I think your problem is here: You are not inserting some substring from a each time through the loop, but a itself. Why are you doing it S times? Perhaps you copied code from an example that had a vector of words rather than a single string. In this code, i is counting how many characters are in s.

for(auto M : s) {
    cout << M << endl;
}

Output each element of the set.
There's only one element; the string a.

Removing duplicate words from string

2 Answers2