Understanding Go pprof statistics

Question

I'm learning how Go profilig works. So I do the following example

func Func() {

    var b [][]byte

    for i := 0; i < 8; i++ {
        b = append(b, make([]byte, 8))
    }

}

func BenchmarkFunc(b *testing.B) {

    for i:=0; i<b.N; i++ {
        Func()
    }
}

then I run

go test -run none -bench Func -benchmem -memprofile mem.out

and

go tool pprof -alloc_space mem.out 
(pprof) list Func

What I get is

Total: 1.61GB
ROUTINE ======================== test-mem/somepkg.BenchmarkFunc in .../work/go/src/test-mem/somepkg/somepkg_test.go
         0     1.61GB (flat, cum)   100% of Total
         .          .     14://
         .          .     15:
         .          .     16:func BenchmarkFunc(b *testing.B) {
         .          .     17:
         .          .     18:   for i:=0; i<b.N; i++ {
         .     1.61GB     19:           Func()
         .          .     20:   }
         .          .     21:}
ROUTINE ======================== test-mem/somepkg.Func in .../go/src/test-mem/somepkg/somepkg.go
    1.61GB     1.61GB (flat, cum)   100% of Total
         .          .     42:func Func() {
         .          .     43:
         .          .     44:   var b [][]byte
         .          .     45:
         .          .     46:   for i := 0; i < 8; i++ {
    1.61GB     1.61GB     47:           b = append(b, make([]byte, 8))
         .          .     48:   }
         .          .     49:
         .          .     50:}

Why I get 1.61GB allocation? Isn't it too much? How can I check what is one function run memory allocation?

`Why I get 1.61GB allocation?` because the profiler has run many times (remember b.N ?) to collect stats and this is what he found. `Isn't it too much?` depends what you are looking to optimize, memory consumption, cpu performance. In current case, no it is not, it happens your code is such a micro benchmark that the profiler got to run many many times before it can get correct stats. `How can I check what is one function run memory allocation?` can you rephrase please ? — , Dec 13 '19 at 11:24
It supposedly isn't. What stats does that sample `go test` run printed you? I mean, what number of iterations the benchmarking engine deemed suitable to run? The next thing to consider is that `pprof` counts actual memory allocaton, and it's quite possble that `make([]byte, 8)` actually allocated more than 8 bytes, and also `append` actully grows the slice using an algorythm more tricky than you might though it is: until it hits certain memory size, it actually allocates ×2 the space you've tried to append—in order to try to inhibit _reallocations_ on future appends. — kostix, Dec 13 '19 at 11:25
… Also note that when that guessing fails, and it must reallocate, it allocates a _new_ memory chunk which is larger than the original and copies the existing elements over there. And while the old backing array becomes garbage, the new whole array _is_ an allocation. All-in-all, the runtime ends up allocating much more than what you could have calculated by applying simple-minded logic (like `(8 + sizeof(sliceHeader)) × b.N`). — kostix, Dec 13 '19 at 11:27

Understanding Go pprof statistics

0 Answers0