0

I have big project written in c# wpf, also I have continuous integration in jenkins that build the project and run unit test after each commit.

The build is incremental. I have problem that the CI is so slow because of the unit test

Time taken to build the whole solution: 7min

Time taken to run all the unit test( ~350 unit test): 10min I use this script:

for /D %%f in (*.Tests) do call :run_mstest %%f

echo exit flag is %EXIT_FLAG% 
echo ---------------------------------------------------------------------- 
if "%EXIT_FLAG%" == "0" (
    echo Unit Test ended OK. 
) else ( 
    set ERROR_CODE= 1
    echo Unit Test ended NOT OK! 
)
goto :end
:run_mstest


    Echo Running %1 
    set current_test=%1
    set current_test_result_name=%current_test:.=%Result
    set dll_name=%1\bin\%Configuration%\%1.dll
    if not exist "%dll_name%" (
        set dll_name=%1\bin\%1.dll
    )

    if not exist "%dll_name%" (
        goto :error1
    ) else (
        echo call vstest.console.exe" /PLATFORM:X64 %dll_name%
        call vstest.console.exe" /PLATFORM:X64 %dll_name%
        if ERRORLEVEL 1 (

            echo FAILED
        ) else (
             echo PASSED
        )
    )

is there a way to run the test dll only for projects that changed by commit? because my build is incremental I want to reduce the time of running those unit tests, how I can do that? Thanks

Zakk
  • 758
  • 3
  • 11
  • 20
  • The time it takes the compiler to build the binaries for the project has no correlation on the amount of time the test runner takes to run the tests. Many test runners offer methods to choose which tests to run based on pre-determined labels, but you'd need to inspect the source code from the previous commit to determine yourself which tests got modified and need to re-run. Plus, what if the test itself didn't change but the code it is testing did? – gunr2171 Nov 26 '18 at 21:04
  • In your code, you already have the power to run or not run whole DLL's, your issue is that you need to look through the git log to determine if the backing project was modified and run the test DLL only if that's the case. No test runner supports that logic. – gunr2171 Nov 26 '18 at 21:07
  • @gunr2171 you correct. Also it sounds not easy task. I saw there is /Parallel argument but I still didn't find good example of it. – Zakk Nov 26 '18 at 21:09
  • Bear in mind that checking in code in one place can break tests in a different place. For example, if I add a new value to an enum, this may cause a unit test failure somewhere completely different: to remind me that I need to map that new value in a method which maps those values to a third party API's enum. – Richardissimo Nov 26 '18 at 22:10
  • I would take a look at why the tests take that long, but if your compile takes 7 minutes, it indicates either a lot of code (implying a lot of tests), or a slow machine. Either of those would lead to the tests needing a fair amount of time to execute. So unless you can break off part of that system into a subsystem which is independently built and tested (e.g. into a NuGet package) or buy a faster build server, then you have to decide whether to live with it, or switch off some tests. – Richardissimo Nov 26 '18 at 22:16

0 Answers0