Python constant folding with labels

Question

I would like to do something that is similar to constant folding using Python.

Python has a convenient built function, eval(), so that constant only equations can be easily folded in by applying eval().

Example:

s = '4 + (5) * 2'
reduced_s = str(eval(s))  # '14'

However, constant folding I want to implement should handle non-constant labels.

Examples:

s = '_tbl + (2) + (2) * 4' 
should be folded to '_tbl + 10'.

If constant parts are separated by a label, for example (s = '2 + _tbl + 4'), it should still produce '_tbl + 6' (or '6 + _tbl).

I have written a constant folding routine using "C" years ago. It was not a small routine as I needed to construct a tree and evaluate the precedence of operators.

As Python is much more powerful langauge than "C", before doing the same thing using Python, I would like to seek other people's wisdom.

Your insights to this challenge are greatly appreciated.

What are you trying to get out of this? Do you just mean you want to transform the string `'_tbl + (2) + (2) * 4'` into the string `'_tbl + 10'`? What are you using the strings for? — BrenBarn, Feb 02 '15 at 04:47
Yes, Python does supply an `eval` function; there is also `exec`. But that doesn't mean you should use them. :) In general, it's highly recommended to organize your code so that you don't need to use `eval`. Eg you can compute constant values in the script's global context so that they aren't computed each time you call a function; you can make lists or dicts of lambdas; you can use [partial functions](https://docs.python.org/2/library/functools.html#functools.partial); etc. — PM 2Ring, Feb 02 '15 at 05:01
If you `eval` strings whose full content is only known at run-time you have to be _very_ careful to ensure that dangerous code isn't executed. See [Eval really is dangerous](http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html) by SO regular Ned Batchelder. But if you _really_ want to use an `eval`-ish approach, rather than the techniques I mentioned in my previous comment, you may be able to use [ast.literal_eval()](https://docs.python.org/2/library/ast.html#ast.literal_eval) instead. — PM 2Ring, Feb 02 '15 at 05:02
The OP should use the `ast` module parse the expression into a AST and then applying their constant folding to the AST and then convert the AST back into Python code. — Dan D., Feb 02 '15 at 05:04
> What are you trying to get out of this? This is something to do with code optimization (data flow analysis). Currently only cases where two cases are literally identical are considered identical. So, '_lbl + (2) + (2)' and '_lbl + (4)' are not. — user2756376, Feb 02 '15 at 18:43
Thank you for educating me how dangerous eval() and exec() could be. — user2756376, Feb 02 '15 at 19:05

score 0 · Answer 1 · answered Feb 02 '15 at 05:01

If the "+" is contained in the eval expression, then it will be difficult that you need to parse the expression, but if the '+' operation is outside of the string expression, then you can overload the operator "add" to simulate the eval.

class MyString(str):
    def __init__(self):
        self.mIntList = []
        self.mStringList = []

    def __add__(self, other):
        result = 0
        try:
            result = eval(other)
        except:
            self.mStringList.append(other)
        else:
            self.mIntList.append(result)
        finally:
            return self

    def __str__(self):
        return str(sum(self.mIntList)) + " ".join(self.mStringList)

if __name__ == '__main__':
    s1 = MyString()
    s1 = s1 + "_table" + "(2) + (2) * 4"
    print s1
    s2 = MyString()
    s2 = s2 + "2" + "table" + "2*4"
    print s2

output is

10_table
10table

score 0 · Answer 2 · answered Feb 02 '15 at 07:19

As pointed out in comments comment, using ast module sounds a good idea. Below i am giving a sample to separate the non-constant label and to evaluate the rest of the expression. Hope, this will give some positive direction.

import ast

label = None

class MyTransformer(ast.NodeTransformer):

    def visit_Name(self, node):
        global label

        if(node.id != "evaluated"):
            label = node.id
            return ast.Num(0)
        return node


expression = '_tbl + (2) + (2) * 4' 

node = ast.parse("evaluated=" + expression)
node=MyTransformer().visit(node)
node = ast.fix_missing_locations(node)
eval(compile(node, '<string>', 'exec'))

print(label, str(evaluated))

Thank you for introducing me the ast module. It is interesting to know such module is available in Python. It generate a tree after parse() is invoked. However, there is no constant folding method to apply within the ast module and it is still done by eval(). If that is the case, I don't see much difference to a crude approach, using re.search('[_A-Za-z][_A-Za-z0-9]*', exp) to detect if a label is there. Split the exp if a label is there then apply eval() to a string that is after the label. For my case, a label always comes first if there. So, this would work. — user2756376, Feb 03 '15 at 01:38

Python constant folding with labels

2 Answers2