0

I am using antlr 2.7.4 to create a lexer. I am stuck at the following case:

  • if Colon (':') is followed by characters from class C1, COLON token should be emitted followed by token C1

  • if Colon is followed by Character from character class C2, colon should be taken as part of C2 and token C2 should be emitted.

Assuming Class C1 is {1,2,3} and Class C2 is {A,B,C} then :13 should tokenized as COLON followed by C1. However :AB should be tokenized as C2.

More concretely, I have grammar for a language which has two constructs:

  • Identifier : Type // It has three tokens: IDENT COLON IDENT . Pascall like type annotation
  • :: // this is an identifier. There is a class of character that can be used as identifiers. Colon can be used as identifier provided it is used with other characters of the class

Some examples:

  • myvar : Int // IDENT COLON IDENT
  • :: // IDENT
  • :$ // IDENT
  • &:& // IDENT

A followup question: Is it possible to check if a certain look ahead character belongs to certain Token class ? Any suggestions will be really appreciated.

EDIT

I guess I am the only user of antlrv2. I will be happy to have a solution in antlrv3 and see if I could hack it in antlrv2.

Jamil
  • 2,150
  • 1
  • 19
  • 20
  • Unless you have a compelling reason to use ANTLR v2, I'd go for v3 instead. – Bart Kiers Dec 26 '12 at 19:24
  • Unfortunately I *have* to use antlr2. – Jamil Dec 26 '12 at 19:59
  • You might find out (by the lack of responses) that ANTLR v2 is not used that much anymore. All I can do is recommend you take it up with the powers that be that force you to use this old version and try to convince them to let you use a newer version (assuming that is possible, otherwise, you might be just out of luck). – Bart Kiers Dec 27 '12 at 08:03
  • will be happy to accept a solution in antlrv3. – Jamil Dec 27 '12 at 09:38
  • I'll have a closer look at it this evening (@work now) and might propose a v3 solution (if no one else makes a suggestion before then). – Bart Kiers Dec 27 '12 at 11:50

0 Answers0