0

I would like to just tokenize a text using RSyntaxtText's TokenMaker class. TokenMaker.getTokenList looks promising, however it only works for a single line and I'm a little bit worried that tokenizing a complete file isn't just calling this method for every line of the file (multi-line comments, opening/closing curly-braces, ...).

How would a minimal portion of code look like which properly tokenizes e.g. a Java file?

predi
  • 5,528
  • 32
  • 60
mstrap
  • 16,808
  • 10
  • 56
  • 86
  • RSyntaxTextArea utilizes [`JFlex`](http://jflex.de/) for lexical tokenization. The `TokenMaker` class is just an interface that wraps code generated by it. It is unclear what your intention is, but you can probably completely omit relying on RSTA and go straight to `JFlex` tutorials. – predi Jul 22 '16 at 11:53
  • I'm looking for a *light-weight* solution which can be used for syntax hightlighting of our *custom* SWT text component. Would you still recommend to do that directly with JFlex? I guess I could still reuse the RSTA `.flex` language definitions there? – mstrap Jul 24 '16 at 13:25

0 Answers0