Cannot generate go coverage file

Question

I tried run a sytemTest in this article: https://www.elastic.co/blog/code-coverage-for-your-golang-system-tests

so follow the tips first I create a system test file named main_test.go like this:

func TestSystem(t *testing.T) {
    t.Logf("systemtest mod=%v", *SystemTest)
    if *SystemTest {
        t.Log("runing system test....")
        main()
    }
}

when this unit test is executed, whole main function will be executed

then I build a test birnary :

go test -c -covermode=count -coverpkg ./... -o main.test

and execute the test birnary file in my test environment

./main.test -systemTest  -test.coverprofile ./coverage.cov

because the program will listen and waiting for client's request, so it will not exit unless I exit manunal, which means the coverprofile won't be genarated

so I start a timer to stop the program after 15 seconds... however when program exits , the coverprofile still not genarated

if test not call main, the coverprofile can be genarated normally

See the main function

var mkrtExitWait sync.WaitGroup
var mkrtExitCode int
var mkrtRunning bool = false

func MKrtRun() int {
    mkrtExitWait.Add(1)
    mkrtRunning = true
    mkrtExitWait.Wait()
    return mkrtExitCode
}
func MKrtExit(code int) {
    if !mkrtRunning {
        os.Exit(code)
    } else {
        mkrtRunning = false
        mkrtExitCode = code
        mkrtExitWait.Done()
    }
}


func main() {
    // listen and serve code 
    ......

    if *SystemTest {  // a command flag 
        go func(){
            time.Sleep(time.Second * 10)
            MKrtExit(0)
        }()
    }
    MKrtRun()
}

I tried some ways to genrate coverage file as followed, but it's not working:

1. send a client request to tell test server to execute os.Exit(0) when program is running

2. send a client request to tell test server to execute panic() when program is running

3. kill the process to force exit the program

What's the problem?

How can I generate coverage file?

Answer 1

I figured out the reason why coverprofile cannot be genarated ..

the main coroutine is started by test binary, it will wait util all test tasks finished, and finally genarate coveragefile. main function test is one of tasks, so if main function execute os.Exit(0), coverage file will not be genarated. and In my test program, when main function test task was over, there remains some other test tasks to execute, one of them was blocked by an I/O event without timeout, so it has nothing to do with the main function test task.

Answer 2

What solved it for me (and mentioned as part of @darkwing's answer) is the flags order.

When I placed the -test.coverprofile=coverage.cov first, it worked, and the coverage.cov file was created.

Note: I had to add the = sign between the flag name and value.

Also I didn't need run it in a different go routine.

Answer 3

The OP hinted at an answer but didn't really give one. The key is that in order for a coverage profile to be written the program needs to return execution to TestMain and cannot just call os.Exit(0)

So all that really needs to be done is a little handling code with a return from the main goroutine. Below is example code, it is run after compiling the test binary using:

go test -c -covermode=count -cover -coverpkg ./...

And then it is ran like this (the quotes were necessary in powershell on Windows but not cmd):

testmain.test.exe "-test.coverprofile" coverage.cov systemTest

I have found that for some reason the coverage file is only generated if my custom flag systemTest comes after the -test.coverprofile coverage.cov Here's a simple example:

main.go:

func main() {
    c := make(chan os.Signal, 1)
    signal.Notify(c, os.Interrupt)
    done := make(chan struct{})


    go func() {
        <-c
        done <- struct{}{}
    }()

    go func() {
        for {
            // this is where your real program runs
            fmt.Println("printing and sleeping")
            time.Sleep(1 * time.Second)
        }
    }()
    <-done
    return
}

main_test.go:

func TestMain(t *testing.T) {
    run := false

    for _, arg := range os.Args {
        switch {
        case arg == "systemTest":
            run = true
        }
    }

    if run {
        main()
        fmt.Println("returned from main")
    }
}

Inspiration: https://www.cyphar.com/blog/post/20170412-golang-integration-coverage and https://www.elastic.co/blog/code-coverage-for-your-golang-system-tests

Answer 4

System testing is usually used when you are running some external tests on your testing

This would mean you need to modify your main program to exit on certain intrrupts. SIGINT is most apt signal for your main program to exit as it confirms with POSIX signal compliance

This can be done by adding following

        // Handle graceful shutdown of http server via OS Interrupt
        quit := make(chan os.Signal)
        signal.Notify(quit, os.Interrupt)

        // Block a goroutine on Interrupt channel and close
        // http server to exit gracefully
        go func() {
            <-quit
            log.Info("receive interrupt signal")
            MKrtEixt(0)
        }() 

So then your testing steps would be

1. Bring up system-under-test
2. Run external tests
3. Issue SIGINT to system-under-test, once tests complete
4. Collect the coverage files