Python slimit minimizer unwanted warning output

Question

from slimit import minify


if __name__ == "__main__":
    print("start")

    # Normally, I pass real JavaScript. For this issue, an empty string reproduces problem.
    minify("", mangle=True)

    print("exit")

This triggers the following console output.

start
WARNING: Couldn't write lextab module <module 'slimit.lextab' from '/Users/kurtostfeld/samba/wrapad/venv/lib/python2.7/site-packages/slimit/lextab.pyc'>. Won't overwrite existing lextab module
WARNING: yacc table file version is out of date
WARNING: Token 'IMPORT' defined, but not used
WARNING: Token 'BLOCK_COMMENT' defined, but not used
WARNING: Token 'ENUM' defined, but not used
WARNING: Token 'EXTENDS' defined, but not used
WARNING: Token 'LINE_COMMENT' defined, but not used
WARNING: Token 'LINE_TERMINATOR' defined, but not used
WARNING: Token 'CONST' defined, but not used
WARNING: Token 'EXPORT' defined, but not used
WARNING: Token 'CLASS' defined, but not used
WARNING: Token 'SUPER' defined, but not used
WARNING: There are 10 unused tokens
WARNING: Couldn't create <module 'slimit.yacctab' from '/Users/kurtostfeld/samba/wrapad/venv/lib/python2.7/site-packages/slimit/yacctab.pyc'>. Won't overwrite existing tabmodule
exit

These warnings are flooding my application console output. How can I use minify without generating warnings?

I'm using Python 2.7.12, and what is currently the latest library versions: slimit 0.8.1, ply 3.10.

snoob dogg · Answer 1 · 2018-08-16T21:01:58.950

5

According to this issue on Github, slimit depends of the ply package. After few tries, it seems that theses warnings appear since version 3.8 of ply. . You could update ply to 3.6 which is the last version that doesn't bring these messages :

pip uninstall ply -y && pip install ply==3.6

It solved my problem.

UPDATE Install a sooner version of ply was really a bad work around since some of my tests were failing. Original slimit version seems not maintained well so I suggest to update to a newer version, metatoaster did a good job to improve it and fixed that problem of warning message. The solution for me was to uninstall slimit and then install it's version:

pip install git+https://github.com/metatoaster/slimit.git#egg=slimit

FINAL UPDATE In fact, slimit seems not maintened anymore and its successor is called calmjs, there is few differences but it is really more stable and don't shows these annoying warning message. See: https://github.com/calmjs/calmjs.parse

edited Aug 16 '18 at 21:01

answered Aug 02 '18 at 17:47

snoob dogg

2,491
3
31
54

ply 3.4 is from 2011. ply 3.11 is from 2018. 3.11 is the newer release: https://pypi.org/project/ply/#history – clay Aug 02 '18 at 21:42
Where did you see that `slimit` is not maintained anymore? – Be Chiller Too Dec 17 '18 at 09:18
1

@BeChillerToo don't remember honnestly. I probably did a fake news by mistake, calmjs seems more active anyway – snoob dogg Dec 17 '18 at 11:21
@snoobdogg in fact you're probably right. I checked the Github page pour slimit, it has some recent commits, but the last release is from 2013... – Be Chiller Too Dec 17 '18 at 13:09
1

The above url does not work any more, but the following one currently solves the issue: https://github.com/rspivak/slimit – jciloa Mar 02 '20 at 13:06
@jciloa calmjs is a better alternative to slimit. last commit on the repos of your url is from august 2018. Last commit from calmjs repos is janunary 2020 – snoob dogg Mar 02 '20 at 13:15

score 3 · Answer 2 · answered Dec 17 '18 at 09:31

Switching versions did not change anything for me, I found another workaround: simply delete (or move, if you want to be cautious) the mentioned files (yourpython/site-packages/slimit/yacctab.py and yourpython/site-packages/slimit/lextab.py).

I believe the module will re-create these files and stop bothering you with warning messages.

score 2 · Answer 3 · answered Jul 01 '17 at 18:40

Slimit uses ply under the hood, which uses logging from stdlib. AFAICS slimit does not allow you to pass the logging parameters that ply's lex and yacc expect.

While you therefor can't (directly) access ply's logging, you should be able to suppress those messages my raising the global logging level:

import logging
...
logging.disable(logging.CRITICAL)
minify("", mangle=True)
logging.disable(logging.NOTSET)

jciloa · Answer 4 · 2020-03-02T13:43:32.807

1

You can use this parser, too: https://github.com/PiotrDabkowski/pyjsparser

It works and is easy to use. It does not handle comments though. (Neither does calmjs seems to handle comments fully: Its parse function has a parameter to indicate that you want the comments, but as of now, some comments seem to get lost.)

edited Mar 02 '20 at 13:43

answered Mar 02 '20 at 13:19

jciloa

1,039
1
11
22

score 0 · Accepted Answer · answered Jul 05 '17 at 21:01

Here was the solution that I went with. I made a custom variant of two slimit functions that pass an extra errorlog=ply.yacc.NullLogger() call to the ply.yacc.yacc function.

class SlimitNoLoggingParser(Parser):
    """
    This is a simple customized variant to slimit.parser.Parser.

    The only difference is that this passes a errorlog=ply.yacc.NullLogger() to ply.yacc.yacc to suppress unwanted
    stderr logging output.
    """

    def __init__(self, lex_optimize=True, lextab=lextab,
                 yacc_optimize=True, yacctab=yacctab, yacc_debug=False):
        self.lex_optimize = lex_optimize
        self.lextab = lextab
        self.yacc_optimize = yacc_optimize
        self.yacctab = yacctab
        self.yacc_debug = yacc_debug

        self.lexer = Lexer()
        self.lexer.build(optimize=lex_optimize, lextab=lextab)
        self.tokens = self.lexer.tokens

        self.parser = ply.yacc.yacc(
            module=self, optimize=yacc_optimize,
            errorlog=ply.yacc.NullLogger(),
            debug=yacc_debug, tabmodule=yacctab, start='program')

        # https://github.com/rspivak/slimit/issues/29
        # lexer.auto_semi can cause a loop in a parser
        # when a parser error happens on a token right after
        # a newline.
        # We keep record of the tokens that caused p_error
        # and if the token has already been seen - we raise
        # a SyntaxError exception to avoid looping over and
        # over again.
        self._error_tokens = {}


# This is a simply variant of slimit.minify that suppresses unwanted noisy stderr logging output.
def warning_free_minify(text, mangle=False, mangle_toplevel=False):
    parser = SlimitNoLoggingParser(lex_optimize=False)
    tree = parser.parse(text)
    if mangle:
        mangler.mangle(tree, toplevel=mangle_toplevel)
    minified = ECMAMinifier().visit(tree)
    return minified

Python slimit minimizer unwanted warning output

5 Answers5