2

TeamCity is showing a certain number of tests in the main page: Main page

However, if I enter the test itself then it is showing a different number: Test page

I have checked, and it appears that the number shown inside the test (78) is the real number of tests that ran. Why is it showing a different number in the main page?

edit: It appears that in the overview tab it is also showing 77 at the top, but stating that 78 tests have passed, as they should. Why is the number above the only one that is not matching? Overview Tab

Kfir Cohen
  • 153
  • 1
  • 3
  • 13
  • I've seen this before. Any chance two of your tests have the same name? I think TC changed in 8.0 to show them like this. (Which may well be an unreported bug.) – sferencik Feb 23 '16 at 17:27
  • @sferencik I am running multiple files, which share 1 function in common. But this command is coming from 3 different files, so I doubt it is the cause. I will change that function to be different in all files and test again. – Kfir Cohen Feb 24 '16 at 12:25
  • In the screenshot above, in the build whose caption is "Tests passed: 77", when you open the build details, "Tests" tab, you'll presumably see a list of 77 passed tests. I'm assuming you're expecting 78, and you can produce a list (e.g. by analysing the source code of your test suite). Compare the two sets. What's missing in the "77" set? – sferencik Feb 24 '16 at 14:13
  • @sferencik All 78 tests are showing in the "Tests" tab. Nothing is missing there.. In the "Overview" tab I can also see "Tests passed: 77", and then just below "78 tests passed" (view updated image above). I am not sure how to check which one he is not counting, since all of the lists show 78 tests. Only the count in the main page is missing 1. Any idea how to check which one he is not counting? – Kfir Cohen Feb 24 '16 at 16:00
  • Now when you take the 78 test names from the Tests tab... Are all the names (as TeamCity has them) unique? – sferencik Feb 25 '16 at 15:32
  • @sferencik There is one test whose name isn't unique, and is showing 3 times (as expected), each time from a different file. Could this be the cause? – Kfir Cohen Feb 28 '16 at 08:04
  • Yes. I'll post this as an answer, with more detail. – sferencik Feb 29 '16 at 11:14
  • Actually, I have more questions... Tests with identical names are only counted once. In your case, if you open the *Tests* tab and look up the non-unique name, you should see the test only once (and TC will say that there were "*3 runs*" of that test). Can you confirm? Also, you still haven't said which is the correct number: 77 or 78 or some other number? In the *Tests* tab, upper-right corner, change to Show *All* items. How many tests do you see? – sferencik Feb 29 '16 at 11:24
  • @sferencik Thank you for the assistance. As I have said in my post, 78 is the real number of tests. If I look up the non-unique name, it is showing 3 times, ordered #1, #27 and #53 (the order of their run). I am seeing 78 tests when I switch to _Show All Items_. The _Tests_ tab is showing the correct number. Only the count in the main page (picture 1) is showing the wrong number. – Kfir Cohen Feb 29 '16 at 14:59
  • Now that's puzzling. If they really had the same name (the *fully qualified* name, incl. package name, class name), TC would consider them equal (and would say you've run the test 3 times). Maybe I've mislead you and the three tests actually *differ* in the fully-qualified name? In which case the "name clash" idea is a red herring... – sferencik Feb 29 '16 at 16:14
  • @sferencik They do differ slightly since they are coming from 3 different files, thus they have a different package name. The function name is the only thing that is the same. Back to square 1? – Kfir Cohen Mar 01 '16 at 12:27
  • Yeah, seems like it. Raise it with JetBrains directly at https://youtrack.jetbrains.com/issues. – sferencik Mar 01 '16 at 14:04
  • Was this issue ever raised? Did you solve the problem? – duncanhall Aug 01 '16 at 11:12

1 Answers1

1

The test logs of junit type are being truncated for some reason. While I have no fix form that, you can try to change the test output to teamcity Service Messages style. PHPUnit has the --teamcity option for example.

pawzar
  • 113
  • 1
  • 5