Inject new lexemes in a yacc Rule

Question

I have the following grammar (this a simplified one):

S -> EXPR

EXPR -> ITEM addop EXPR
EXPR -> ITEM

ITEM -> num
ITEM -> ident

having:

num: a floating point number
ident: a string representing an identifier
addop: +

I am using PLY library for python, and have the following code:

def p_L_S(self,p):
  ''' S : EXPR'''
  p[0] = p[1]

def p_L_EXPR_1(self,p):
  ''' EXPR : ITEM addop EXPR'''
  p[0] = p[1] + p[2]

def p_L_EXPR_2(self,p):
  ''' EXPR : ITEM'''
  p[0] = p[1]

def p_L_ITEM_1(self,p):
  ''' ITEM : num '''
  p[0] = float(p[1])

def p_L_ITEM_2(self,p):
  ''' ITEM : ident '''
  p[0] = value_of_expr_associated_to_ident(p[1])

[...]

in the last function (p_L_ITEM_2) I would like to interprete the string associated to p[1] (which is an expression recognized by the grammar) without launching another parsing. Today, the function value_of_expr_associated_to_ident launches a new parsing (calling parse method) of the expression associated to ident. But this gives really poor performances, even if it works.

Is there a way to send to the parser the lexems associated to the expresion associated to the ident to avoid having to start a new parsing ?

I don't know if this is clear, and if not I will try to clarify.

Thanks a lot. Sam

score 1 · Accepted Answer · answered Jan 04 '16 at 17:35

If you are trying to do some sort of lazy evaluation in a functional language, read on. That's not as easy as it looks, and I haven't provided anything more than a rough idea about the approach.

If the values associated with identifiers are plain strings representing expressions in the language, then a recursive call to parse is what you need to do.

But it seems like it would be worthwhile at least caching the resulting parse tree (which means that your parser needs to create a parse tree, rather than doing an immediate evaluation). Alternatively, you could parse the string value into a parse tree when you assign it to the variable.

However you accomplish the recursive parse, you need to somehow deal with infinite recursion, as exemplified by the value of r being r (or r + 1).

I like the idea of caching the parse tree... I will look for a way to sdo so. Thanks. — Samfaitmal, Jan 04 '16 at 19:31
For information : I have cached the lexemes associated to expressions stored for each 'ident'. It has increased performances by 20%. — Samfaitmal, Jan 06 '16 at 10:58

Inject new lexemes in a yacc Rule

1 Answers1