0

I have a Winforms application (MVP architecture) using .NET 4.6.2 and C#7 enabled.

In TFS 2017 we have 2 build definitions for the app: 1. a Gated checkin - which works all the time. 2 A Scheduled build that runs at 11PM Mon-Fri. - which doesn't work.

The build definitions are pretty much identical (nuget restore, VS Build with 2017, run tests...) and build the same solution. The only difference is that the scheduled overnight build additionally runs code analysis and code coverage is enabled. Platform & Config is Any CPU | Release.

The scheduled overnight build fails on 8 tests. Some tests invoke async methods in the SUTs but others are pretty vanilla, so I don't see any pattern.

The test projects depend on the following nuget packages:

  • MSTest.TestAdapter 1.1.18
  • MSTest.TestFramework 1.1.18
  • Moq 4.2.1507.118

The only thing I see that is different is that the Gated checkin is queued under the developer's name, whereas the scheduled evening build is kicked off by Microsoft.TeamFoundation.System

TL:DR version: How can I diagnose why the tests won't run at night, yet work just fine with the Gated checkin?

Scott
  • 924
  • 3
  • 8
  • 28
  • Are they use the same build agent? Are there any logs to track that? – Andy Li-MSFT Sep 22 '17 at 03:14
  • Hi Andy, sorry for the delay in replying. They do use the same build agent, and I've had a quick look at the logs, and the only thing I see (or rather, recognise as an issue) is the failing tests. – Scott Sep 26 '17 at 09:13

1 Answers1

0

Pleas try below items to narrow down the issue:

  • Check if the definitions are using the same build agent, if not just try with the same build agent.
  • Deploy a new agent for the scheduled build definition.
  • Keep all the same for both build definitions except the Gated Check-in option. That means disable code analysis and code coverage for the scheduled build definition.
  • Also enable the Gated Check-in for the scheduled definition.

Then check that again.

According to this thread, there is an issue for the scheduled build on earlier version of TFS 2017. So, if you are using the version earlier than Update1 RC2, then you can try upgrading to the latest version, then try it again.

For version check, please refer to this link : https://blogs.msdn.microsoft.com/tfssetup/2013/11/21/what-version-of-team-foundation-server-do-i-have/

Andy Li-MSFT
  • 28,712
  • 2
  • 33
  • 55
  • OK, I have done what you say. Will wait until tomorrow morning before I get back to you. Can you tell me any reason why manually queuing the scheduled evening build with code coverage enabled & code analysis works fine? Thanks for your assistance, much appreciated. – Scott Sep 26 '17 at 09:13
  • Hi, I removed the Code analysis and the code coverage settings, and the build still failed due to failing tests. Again, when I queue the build manually in TFS the build succeeds. – Scott Sep 27 '17 at 11:44
  • @Scott What about creating a new scheduled build definition? also try to disable the test steps in the build definition, then try it again. And what's the exact version of TFS 2017? check that according to [this link](https://blogs.msdn.microsoft.com/tfssetup/2013/11/21/what-version-of-team-foundation-server-do-i-have/). There is a similar issue on earlier version, check [this thread](https://developercommunity.visualstudio.com/content/problem/18956/only-scheduled-tfs-builds-failing-with-one-or-more.html). So, If you are using older version, just try to upgrade to the latest version. – Andy Li-MSFT Sep 28 '17 at 01:56
  • Andy, extremely sorry for delay in reply - I didn't see the notification. Will try creating a new scheduled build def. The version of TFS is 15.117.26714.0 (Tfs2017.Update2) – Scott Oct 04 '17 at 08:15