Stack Overflow
Stack Overflow

Questions tagged [boost-spirit-lex]

60 questions
3
votes
1 answer

How to use boost::spirit::qi with a std::vector instead of std::string

In an application, I basically want to have a "pre-parsing" phase where I adjust the token stream before a Qi parser can see it. One way to do this would be to have some kind of "lexer adaptor" which is constructed from a lexer and is itself a…
Chris Beck
  • 15,614
  • 4
  • 51
  • 87
3
votes
1 answer

Boost Spirit lexer states cross pollinate

I am trying to use lexer states to do context specific parsing, but it seems that different lexer states do cross-pollinate. Here is a very basic example #include #include #include…
Anton Autushka
  • 31
  • 1
3
votes
1 answer

boost::spirit::lex, how does one generate a end of file token?

The question is rather simple, I've written a lexer, using boost::spirit, however I can't seem to find a way, to generate an EOF token. - So how would one go about doing this?
Skeen
  • 4,614
  • 5
  • 41
  • 67
3
votes
1 answer

Boost.Spirit SQL grammar/lexer failure

I have two problems with the following SQL grammar: #define BOOST_SPIRIT_QI_DEBUG #include #include #include #include…
Pierre T.
  • 380
  • 1
  • 13
3
votes
2 answers

how to get rid of escape character in a token with spirit::lex?

I want to tokenize my own extension of SQL syntax. This involves recognizing an escaped double quote inside a double quoted string. E.g. in MySQL these two string tokens are equivalent: """" (the second double quote acts as an escape character) and…
coproc
  • 6,027
  • 2
  • 20
  • 31
3
votes
1 answer

How to combine boost::spirit::lex & boost::spirit::qi?

I have a lexer and based on that lexer I now want to create a grammar that used the tokens generated by this lexer. I tried adapting some examples that I found and now I have something that compiles and works at least a little bit, but one of my…
Tobias Langner
  • 10,634
  • 6
  • 46
  • 76
3
votes
1 answer

Is there a way to match the content of a spirit::lex string token as a literal in a spirit::qi grammar

I'm writing a DSL and using a Boost Spirit lexer to tokenize my input. In my grammar, I want a rule similar to this (where tok is the lexer): header_block = tok.name >> ':' >> tok.stringval > ';' >> tok.description >> ':' >> tok.stringval >…
Tim Ruddick
  • 1,375
  • 16
  • 24
3
votes
1 answer

Spirit Lex: Which token definition generated this token?

Sorry if this is a newbie question, but I need to know which token definition produced a certain token. When I print the token ID, I just get an integer. I need to know which regex generated this token. Edit: Here's how I define my tokens: …
Haitham Gad
  • 1,529
  • 2
  • 13
  • 23
3
votes
2 answers

Boost spirit lex write token value back to input stream

I'm wondering if there's a way in boost::spirit::lex to write a token value back to the input stream (possibly after editing) and rescanning again. What I'm basically looking for is a functionality like that offered by unput() in Flex. Thanks!
Haitham Gad
  • 1,529
  • 2
  • 13
  • 23
2
votes
1 answer

Cannot get Boost Spirit grammar to use known keys for std::map<>

I seem to be experiencing some mental block with Boost Spirit I just cannot get by. I have a fairly simple grammar I need to handle, where I would like to put the values into a struct, that contains a std::map<> as one of it's members. The key names…
nvanwyen
  • 81
  • 1
  • 4
2
votes
1 answer

Converting a Boost Spirit Lex semantic action to Phoenix - How to access _val?

I wrote a semantic action for my Boost Spirit Lexer to convert escape sequences in strings to what they stand for. It works perfectly and I want to convert it to a Boost Phoenix expression, but can't get that one to compile. Here is what works: //…
Felix Dombek
  • 13,664
  • 17
  • 79
  • 131
2
votes
1 answer

Case-insensitive keywords with boost::spirit::lex

Is there a way to recognize specific patterns case-insensitively? E.g. if I have literal_bool = L"True|False"; this->self.add(literal_bool, TokenId_LiteralBool); How can I match true, TRUE, tRuE while avoiding to write [Tt][Rr][Uu][Ee] for each…
Felix Dombek
  • 13,664
  • 17
  • 79
  • 131
2
votes
1 answer

How to make Boost.Spirit.Lex token value be a substring of matched sequence (preferably by regex matching group)

I'm writing a simple expressions parser. It is build on a Boost.Spirit.Qi grammar based on Boost.Spirit.Lex tokens (Boost in version 1.56). The tokens are defined as follows: using namespace boost::spirit; template< typename lexer_t > struct…
Adam Badura
  • 5,069
  • 1
  • 35
  • 70
2
votes
1 answer

Spirit X3 parser start state?

I've been going through the Boost.Spirit X3 documentation I've been able to find---which isn't much---and think I would like to use this for my next parsing project. Notably I have never used Boost.Spirit Classic or V2, but have used flex/bison and…
Zac
  • 876
  • 1
  • 8
  • 18
2
votes
1 answer

Case Insensitive String Comparison of Boost::Spirit Token Text in Semantic Action

I've got a tokeniser and a parser. the parser has a special token type, KEYWORD, for keywords (there are ~50). In my parser I want to ensure that the tokens are what I'd expect, so I've got rules for each. Like so: KW_A = tok.KEYWORDS[_pass = (_1 ==…
Liam M
  • 5,306
  • 4
  • 39
  • 55
1
2
3 4