7

Does anyone know where a FLEX or LEX specification file for Python exists? For example, this is a lex specification for the ANSI C programming language: http://www.quut.com/c/ANSI-C-grammar-l-1998.html

FYI, I am trying to write code highlighting into a Cocoa application. Regex won't do it because I also want grammar parsing to fold code and recognize blocks.

Jed Smith
  • 15,584
  • 8
  • 52
  • 59
pokstad
  • 3,411
  • 3
  • 30
  • 39

3 Answers3

6

Lex is typically just used for tokenizing, not full parsing. Projects that use flex/lex for tokenizing typically use yacc/bison for the actual parsing.

You may want to take a look at ANTLR, a more "modern" alternative to lexx & yacc.

The ANTLR Project has a Github repo containing many ANTLR 4 grammars including at least one for Python 3.

Laurence Gonsalves
  • 137,896
  • 35
  • 246
  • 299
  • So this will both scan and parse? Awesome, I've heard of this but wasn't sure of how mature it was. Besides spitting out C-Code, do you know how well it plays with Objective-C? Any option to produce diagrams (like railroad diagrams or finite state machine diagrams)? – pokstad Nov 14 '09 at 01:03
  • Yes, ANTLR does both scanning and parsing. It's been a while since I last used it, but I remember liking the fact that the scanning and parsing syntaxes were extremely similar. It apparently even has explicit support for Objective-C these days (in addition to C, C++, Java, C#, and Python). I've never used it, but you might want to try ANTLRWorks (http://antlr.org/works/index.html) for diagrams. – Laurence Gonsalves Nov 14 '09 at 01:32
3

grammar.txt is the official, complete Python grammar -- not directly lex compatible, but you should be able to massage it into a suitable form.

ephemient
  • 198,619
  • 38
  • 280
  • 391
  • I have seen that but I noticed it's very different from the lex syntax I've seen. I'm not partial to flex/lex, just looking for something that works with a grammar parser. Do you have any recommendations for a lexical analyzer using that grammar file? – pokstad Nov 14 '09 at 00:57
0

Have you considered using one of the existing code highlighters, like Pygments?

Ned Deily
  • 83,389
  • 16
  • 128
  • 151