1

In my grammar, I have a whitespace token that is sent to the HIDDEN channel:

SP : [ \u00A0\u000B\t\r\n] -> channel(HIDDEN);

I know that I can get the text of a parsed rule, including hidden tokens, with TokenStream#getText(Context). I'd like to have all whitespace collapsed when I call that.

I also know there's a TokenStreamRewriter for rewriting specific tokens, but I don't see a way to rewrite all of a certain type.

Is there any way to collapse all SP tokens to output as a single space?

NickAldwin
  • 11,584
  • 12
  • 52
  • 67
  • At minimum, this is *almost* a duplicate of http://stackoverflow.com/q/25812615/138304. – Sam Harwell Sep 12 '14 at 18:08
  • Indeed. I asked this first, then realized there was a more general question hiding within; I felt it was different enough that it didn't feel appropriate to edit the entire question. – NickAldwin Sep 12 '14 at 18:10

1 Answers1

1

This can be done by changing the rule to match one or more whitespace characters (+) and specifying an action:

SP : [ \u00A0\u000B\t\r\n]+ { _text = " "; } -> channel(HIDDEN);
NickAldwin
  • 11,584
  • 12
  • 52
  • 67