1

Trying to use PhpStorm inspect to tell a story of our code quality and track improvements. It's working well and I have a script that daily pulls from git, runs the inspection and graphs everything. Yay.

Here's the key line that runs the inspection:

command = "phpstorm.bat inspect " + reponame + " " + reponame + "\.idea\workspace.xml results\ -v2 -d " + pathtosrc

However, I've noticed that the numbers given aren't always consistent.

E.g. today, I ran it three times in a row on the same code repo, without any pulls, code changes or anything, and got different results.

Type         Run 1, Run 2, Run 3
ERROR        531,   531,   530
WARNING      12895, 12893, 12878
WEAK WARNING 18425, 18425, 18424
INFORMATION  152,   152,   152
TYPO         82,    82,    82

I can't fathom a reason for this, but if there is one and I can fix it, I'd like to know. I would have thought running the same tool on the same code would always produce the same result, no?

Mark Mayo
  • 12,230
  • 12
  • 54
  • 85
  • And what if you run Inspect Code from the IDE interactively several times in a row, will it show the same results? – duck_in_hat Feb 26 '21 at 02:21
  • @duck_in_hat interesting, I nearly discarded that as the IDE I have scopes set, which aren't as usable on the command line. But I checked anyway. Inconsistent there too! Type Run 1, Run 2, Run 3 ERROR 654, 529, 528 WARNING 12772, 12302, 12510 WEAK WARNING 15834, 15676, 15984 TYPO 33, 30, 30 – Mark Mayo Feb 26 '21 at 03:38
  • Can you limit inspection to a single file, and somehow print out what errors were found? If you understand what errors are inconsistently shown, you might find a solution. – Zsolt Szilagyi Mar 04 '21 at 20:15

1 Answers1

-1

Try to invalidate caches File > Invalidate Caches / Restart... before inspection

ExploitFate
  • 595
  • 2
  • 9
  • sadly no, tried that three times in a row, 3 different results (in the IDE). And I'm talking wildly different results - >5% in cases. – Mark Mayo Mar 04 '21 at 02:13
  • Could you update the question and add the command which you run to inspect code – ExploitFate Mar 04 '21 at 10:34
  • Added. Although it does the same in the IDE, so not sure it's the command line issue :/ (in IDE I go Code->Inspect Code) – Mark Mayo Mar 04 '21 at 20:12