How can I write go profile to file as it goes?

Question

I am trying to profile a go application and below is the code:

func main() {
    defer pprof.StopCPUProfile()
    f, err := os.Create("./profile.tar.gz")
    if err != nil {
        log.Fatal(err)
    }
    pprof.StartCPUProfile(f)
    ...

I am able to see the file was created on the disk once the application exits. Is there a way to flush the profile to disk without quitting the application? My application is a rest API which will keep running for a long time. I'd like to see profile file during the run time. I may create an internal rest path in the application to flush profile data into file.

I think you can use the [net/http/pprof](https://pkg.go.dev/net/http/pprof) package to serve the profile without writing the profile to disk. — Zeke Lu, Jun 11 '23 at 09:26
Implement the writer interface and flush the contents of whatever you are trying to write to the file itself as the programs fill the buffer (this might help https://stackoverflow.com/questions/48115577/bufio-writer-or-io-writer) — ranu, Jun 11 '23 at 14:09

ranu · Answer 1 · 2023-06-11T15:11:11.127

Reading through the code of pprof.NewCPUProfile it doesn't seem to be able to do that. It only writes (and stops profiling) to whatever writer you have passed if it receives an EOF signal when trying to do an internal readProfile. The function that actually writes data into the file is called build and it is only called three times in the package and one of the times it's from the profileWriter (a goroutine spawned by pprof's NewCPUProfile function.

I'd use the net/http/pprof package and from them export whatever profile file you want.

How can I write go profile to file as it goes?

1 Answers1