15

I know there are string tokenizers but is there an "int tokenizer"?

For example, I want to split the string "12 34 46" and have:

list[0]=12

list[1]=34

list[2]=46

In particular, I'm wondering if Boost::Tokenizer does this. Although I couldn't find any examples that didn't use strings.

Community
  • 1
  • 1
Steve
  • 11,831
  • 14
  • 51
  • 63

4 Answers4

14

The C++ String Toolkit Library (StrTk) has the following solution to your problem:

#include <string>
#include <deque>
#include "strtk.hpp"

int main()
{ 
   {
      std::string data = "12 34 46";
      std::deque<int> int_list;
      strtk::parse(data," ",int_list);
   }

   {
      std::string data = "12.12,34.34|46.46 58.58";
      std::deque<double> double_list;
      strtk::parse(data," ,|",double_list);
   }

   return 0;
}

More examples can be found Here

Note: The parsing process is EXTREMELY fast and efficient, putting stdlib and boost based solutions to shame.

13

Yes there is: use a stream, e.g. a stringstream:

stringstream sstr("12 34 46");
int i;
while (sstr >> i)
    list.push_back(i);

Alternatively, you can also use STL algorithms and/or iterator adapters combined with constructors:

vector<int> list = vector<int>(istream_iterator<int>(sstr), istream_iterator<int>());
Konrad Rudolph
  • 530,221
  • 131
  • 937
  • 1,214
  • 1
    Sexy, but unnecessarily verbose. `vector list(istream_iterator(sstr), istream_iterator());` would do just fine. :) – avakar Jul 17 '09 at 07:32
  • 2
    avakar: oddly, *no*, your code doesn’t work. You either need to use the explicit constructor (as done by me) or include an extra pair of braces around one of the arguments; otherwise, your code will *not* work – instead, this is the declaration of a function prototype called `list` with return type `vector`. Try it out! – Konrad Rudolph Jul 17 '09 at 11:53
  • I meant parentheses, not braces. – Konrad Rudolph Jul 17 '09 at 11:54
  • 1
    Have worked on a similar project where one was forced to split "numbers". If you choose to use stringstream for this it will make your program at least 5-10times slower than when using char array / strings to find and split them that way. At the end just convert to int and it's done. This performance loss was tested by me on gcc 4.3 and VS 2008 on a c2d setup when the function splitting int's was run a few billion times vs the string splitting one. – Milan Jul 17 '09 at 12:36
  • Konrad, indeed it won't work. Kudos for spotting that. I'd need a compiler to point it out to me. – avakar Jul 19 '09 at 13:34
0

What you're looking for is 2 separate actions. First tokenize the string, then convert each token to an int.

Tal Pressman
  • 7,199
  • 2
  • 30
  • 33
0

i am not sure if you can do this without using string or char* because you have to but both numbers and spaces into same set...

ufukgun
  • 6,889
  • 8
  • 33
  • 55