How does pylint remember scores from previous runs?

Question

In a typical pylint run, we get the following output:

Global evaluation
-----------------
Your code has been rated at 9.50/10 (previous run: 8.50/10)

Duplication
-----------

+-------------------------+------+---------+-----------+
|                         |now   |previous |difference |
+=========================+======+=========+===========+
|nb duplicated lines      |0     |0        |=          |
+-------------------------+------+---------+-----------+
|percent duplicated lines |0.000 |0.000    |=          |
+-------------------------+------+---------+-----------+

I wonder how pylint remembers scores from previous runs - In the example above, the score from previous run was 8.5.

I want to implement such a functionality in my own module, and I thought I should first find out how pylint implements the functionality.

I searched for hidden folders where this data could be stored, but found none.

you could put the file in a dict as a key and pickle, if the key exists you know you have used the file before then check its previous value — Padraic Cunningham, Aug 24 '14 at 19:12
Thank you, but are you sure this is how pylint implements it? I could not find any pickled file related to my source code. — Kaushik Pavani, Aug 24 '14 at 19:31
Also, tried to run pylint from a different directory on the same .py file, and yet, pylint seems to remember previous scores. This seems to suggest that pylint is not storing previous scores locally. It seems to have global access to score from the previous run. — Kaushik Pavani, Aug 24 '14 at 19:35

Padraic Cunningham · Accepted Answer · 2014-08-24T20:07:59.220

I have a .pylintrc file in my home directory that includes the following line:

#pickle collected data for later comparisons.
persistent=yes

So it seems pylint does use pickle for comparisons

In the lint.py from the source code:

def make_options():
        return (('ignore',
                 {'type' : 'csv', 'metavar' : '<file>[,<file>...]',
                  'dest' : 'black_list', 'default' : ('CVS',),
                  'help' : 'Add files or directories to the blacklist. '
                           'They should be base names, not paths.'}),
                ('persistent',
                 {'default': True, 'type' : 'yn', 'metavar' : '<y_or_n>',
                  'level': 1,
                  'help' : 'Pickle collected data for later comparisons.'})

The full lint.py source is here

The bit that is most interesting is probably this method:

def close(self):
        """close the whole package /module, it's time to make reports !

        if persistent run, pickle results for later comparison
        """
        if self.file_state.base_name is not None:
            # load previous results if any
            previous_stats = config.load_results(self.file_state.base_name)
            # XXX code below needs refactoring to be more reporter agnostic
            self.reporter.on_close(self.stats, previous_stats)
            if self.config.reports:
                sect = self.make_reports(self.stats, previous_stats)
                if self.config.files_output:
                    filename = 'pylint_global.' + self.reporter.extension
                    self.reporter.set_output(open(filename, 'w'))
            else:
                sect = Section()
            if self.config.reports or self.config.output_format == 'html':
                self.reporter.display_results(sect)
            # save results if persistent run
            if self.config.persistent:
                config.save_results(self.stats, self.file_state.base_name)
        else:
            self.reporter.on_close(self.stats, {})

You will also want to look at the config.py source

def load_results(base):
    """try to unpickle and return data from file if it exists and is not
    corrupted

    return an empty dictionary if it doesn't exists
    """
    data_file = get_pdata_path(base, 1)
    try:
        with open(data_file, _PICK_LOAD) as stream:
            return pickle.load(stream)
    except:
        return {}

Excellent, thanks. I found have a folder '.pylint.d' in my home directory which seems to contain the data I was looking for. — Kaushik Pavani, Aug 24 '14 at 20:00
No worries. I was not sure if that was how it was done but I thought it would be an easy way to do what you want. — Padraic Cunningham, Aug 24 '14 at 20:01
If you try and run the try block in the load_results function as written it will always fail into the except clause and return an empty dict since _PICK_LOAD is not defined. One has to look at the code so see that __PICK_LOAD is 'r' for Python2 and 'rb' for Python3. I suggest changing except: to except IOError:. That works for both Python2 and Python3. — Tom Ekberg, Aug 10 '22 at 15:05

How does pylint remember scores from previous runs?

1 Answers1

Linked