How do you return the next token from a stringstream without operator>>?

Question

How do you get a std::stringstream to return the next token, instead of extracting it into a variable with operator>>?

I've included some examples of why I would want to do this below. I've tried get(), but this just returns the next character.

#include <string>
#include <sstream>

std::string line("abc xyz");
std::stringstring ss(line);

std::string token;
ss >> token;                   // I don't want this
token = ss.next();             // I want something like this

if (ss.next() == "abc")        // So I can do something like this
int x = std::stoi(ss.next());  // Or this

The main reason for this is to produce cleaner code when I don't want to store or reuse the next token. Otherwise, the code is filled with lines ss >> token.

`ss >> token; // I don't want this` Why? What's wrong with this? — DimChtz, Aug 10 '17 at 22:59
Everything that you would do with `ss.next()` can be acheived with `ss >> token`. It may take another step, but it will probably result in cleaner code in the long run. — Easton Bornemeier, Aug 10 '17 at 23:01
@DimChtz I have given examples of why I don't want it in the question. — John Terry, Aug 10 '17 at 23:06
@EastonBornemeier Yes I know, but I'm asking if such a function exists. Many tokenizers offer such a function. I don't agree littering your code with `ss >> token` when you don't want to reuse or store the contents of `token` will make your code cleaner. — John Terry, Aug 10 '17 at 23:08
@JohnTerry Reducing clutter is good. A some people use a `for()` loop like this: `for(std::string item; ss >> item;) { if(item == "abc") /* do stuff */ ; }` Then `item` doesn't pollute the wider scope. — Galik, Aug 10 '17 at 23:11

Benjamin Lindley · Accepted Answer · 2017-08-10T23:22:12.007

Write a function.

template<typename T = std::string>
T next(std::istream& is) {
    T value;
    if (is >> value)
        return value;
    else
        throw an_exception_or_something;
}

Then you can do this:

if (next<>(ss) == "abc")
    do_something();
int x = next<int>(ss);   // note: no need for stoi

Alternatively, return a std::optional, if your implementation supports it.

template<typename T = std::string>
std::optional<T> next(std::istream& is) {
    T value;
    if (is >> value)
        return value;
    else
        return std::nullopt;
}

Then you can decide how you want to handle failure at the call site.

int x = *next<int>(ss);            // undefined behavior on error
int y = next<int>(ss).value();     // throws exception on error
int z = next<int>(ss).value_or(0); // return a default value on error

Now that I understand what he's trying to do, this looks like a nice solution. — Barmar, Aug 10 '17 at 23:14

How do you return the next token from a stringstream without operator>>?

1 Answers1