1

Can Boost's Spirit Lex have multiple semantic actions for a token definition?

Consider this Lexer construction:

namespace bp = boost::phoenix;
this->self = lex::token_def<>("[ \v\f\t\r]+")
    | lex::token_def<>("\n")[++bp::ref(line_no)]
    | lex::token_def<>("\\/\\/")[set_lexer_state("single_line_comment")]
    | lex::token_def<>(".")
    ;

this->self("single_line_comment")
    = lex::token_def<>("\n")
    [++bp::ref(line_no), set_lexer_state("INITIAL")]
    | lex::token_def<>(".")
    ;

However only the state switch goes through, but the line ending in state single_line_comment does not get counted.

If I switch the order, e.g.:

[set_lexer_state("INITIAL"), ++bp::ref(line_no)]

The line ending gets counted and the state switch doesn't go through. Is it possible to have multiple semantic actions there separated by commas or do I have to write a single functor to do state switching and any other action I want?

SiimKallas
  • 934
  • 11
  • 23
  • Can you try explicitly adding "#include "? – llonesmiz Apr 06 '14 at 09:39
  • I didn't specify my includes, but it was already added. – SiimKallas Apr 06 '14 at 09:43
  • 2
    If you are using the same `set_lexer_state` that can be found [here](http://www.boost.org/libs/spirit/example/lex/strip_comments_lexer.cpp), you'll need to change it into a phoenix function [(or maybe use `lex::_state="single_line_comment"`)](http://coliru.stacked-crooked.com/a/6d1b47b282f754ae). If you are not ignore this comment and try to show more of your code. – llonesmiz Apr 06 '14 at 12:10
  • Using lex::_state worked, however for debugging purposes I'd like to have a separate function. Thanks for the phoenix function hint, I'll have a look into that. Feel free to answer this post, I'll mark it as accepted. – SiimKallas Apr 06 '14 at 12:26
  • I've tried to make the phoenix function alternative work, but I think(I am far from sure) that it isn't possible. The problem is that you can't access the context from phoenix. If you can't use `lex::_state="bla"` the combined single functor that you mention in our question seems the more sane approach. – llonesmiz Apr 06 '14 at 13:30
  • 1
    I was planning to put [this](http://coliru.stacked-crooked.com/a/dfcffd592dd44e79) as an answer, but then I realized that it does not work when compiled using -std=c++11. After looking into it for a while I have no idea why it fails, so I put this here just in case it helps (although I doubt it will). I have artificially created lots of semantic actions to try to show several ways in which you can define phoenix functions. In your lazy function "invocations" you can use `lex::_start`, `lex::_end`, lex::_pass`, `lex::_tokenid` and the `_ctx` placeholder I've defined. – llonesmiz Apr 08 '14 at 11:32

0 Answers0