The "no known conversion" from const char* to an iterator error - another take

Question

I'm doing the following:

using namespace boost;
const char* line = // ...
size_t line_length = // ...
// ...
tokenizer<escaped_list_separator<char> > line_tokenizer(
    line, line + line_length,
    escaped_list_separator<char>('\\', ',', '\"'));

Expecting to use the boost::tokenizer constructor

tokenizer(Iterator first, Iterator last,
          const TokenizerFunc& f = TokenizerFunc()) 
  : first_(first), last_(last), f_(f) { }

but GCC 4.9.3 gives me:

no known conversion for argument 1 from ‘const char*’ to ‘__gnu_cxx::__normal_iterator<const char*, std::basic_string<char> >’

Now, I've seen a couple of related questions in which the answer was forgetting to #include <algorithm> - but I have included it. Is there some other missing include, or is it another issue?

@Barry: It is succinct, complete, and not machine-verifiable, but I think it's reasonably verifiable. — einpoklum, Apr 12 '16 at 14:32
It's not complete and verifiable until we can compile and reproduce your error (and then eliminate it and tell you what we did). — Toby Speight, Apr 12 '16 at 17:53

score 1 · Answer 1 · answered Apr 12 '16 at 14:03

1

As compiler error says, there is no way to build iterator from const char*. You can fix it by use of std::string:

std::string line = "some string";
// ...
tokenizer<escaped_list_separator<char> > line_tokenizer(
    line.begin(), line.end(),
    escaped_list_separator<char>('\\', ',', '\"'));

answered Apr 12 '16 at 14:03

CodeFuller

30,317
3
63
79

This is in performance-critical code, so I don't want to start copying my strings into the std::string's buffer. I would use the GSL `string_view` or `string_span` or whatever they call it, but I don't want to include the entire GSL. Could you suggest a workaround? – einpoklum Apr 12 '16 at 14:08
Also, a lot of standard library algorithms take a pair of pointers as an iterator pair, e.g. `std::copy`; so why not this thing? – einpoklum Apr 12 '16 at 14:09
1

This tokenizer constructor takes Iterator (which is std::string::const_iterator actually), so you can't use raw char* pointers in this case. – CodeFuller Apr 12 '16 at 14:15

score 1 · Answer 2 · answered Apr 12 '16 at 14:12

you have to build the token iterators manually if you don't want to use a container as the search space

#include <iostream>
#include <string>
#include <boost/tokenizer.hpp>

int main()
{
    const char xx[] = "a,b,c,d,e,f,g";
    auto line = xx;
    size_t line_length = strlen(line);

    using namespace boost;

    auto f = escaped_list_separator<char>('\\', ',', '\"');
    auto beg = make_token_iterator<char>(line ,line + line_length,f);
    auto end = make_token_iterator<char>(line + line_length,line + line_length,f);
    // The above statement could also have been what is below
    // Iter end;
    for(;beg!=end;++beg){
        std::cout << *beg << "\n";
    }
    return 0;
}

einpoklum · Accepted Answer · 2019-01-15T00:46:16.833

Since you're using boost, you can do:

#include <boost/utility/string_ref.hpp>
// ...
const boost::string_ref line_(line, line_length);
tokenizer<escaped_list_separator<char> > line_tokenizer(
    line_, escaped_list_separator<char>('\\', ',', '\"'));

and that seems to work. Read more about string_ref and the other utilities here.

Of course, if you have an implementation of the Guidelines Support Library, use string_span (a.k.a. string_view) from there (here's one implementation). It's even making it into the standard library probably.

Update: string_view is in the C++ standard in C++17. Now you can write:

#include <string_view>
// ...
std::string_view line_ { line, line_length };
tokenizer<escaped_list_separator<char> > line_tokenizer(
    line_, escaped_list_separator<char>('\\', ',', '\"'));

The "no known conversion" from const char* to an iterator error - another take

3 Answers3