Extend memory when grow slice (logic grow capacity underlying array)

Question

I know (and read in internet - including this resource). That the logic for increasing memory is: if len array less than 1024 - golang multiply array on 2 else multiply len on 1.25 (and we see this in source code no question https://github.com/golang/go/blob/cb2353deb74ecc1ca2105be44881c5d563a00fb8/src/runtime/slice.go#L95) but if i fill slice in cycle i see this behaviour

t := []int{}
z := 0
for i := 1; i < 10000; i++ {
    t = append(t, i)

    if z < cap(t) {
        z = cap(t)
        fmt.Println(" len - ", len(t), " : cap - ", cap(t))
    }
}

len a.k.a. num

len -  1  : cap -  1
len -  2  : cap -  2
len -  3  : cap -  4
len -  5  : cap -  8
len -  9  : cap -  16
len -  17  : cap -  32
len -  33  : cap -  64
len -  65  : cap -  128
len -  129  : cap -  256
len -  257  : cap -  512
len -  513  : cap -  1024
len -  1025  : cap -  1280
len -  1281  : cap -  1696
len -  1697  : cap -  2304
len -  2305  : cap -  3072
len -  3073  : cap -  4096
len -  4097  : cap -  5120
len -  5121  : cap -  7168
len -  7169  : cap -  9216
len -  9217  : cap -  12288

before len 513 - capacity grow x2
len 1025 = 1.25 * len 513 = 1280 - capacity grow x1.25 (ok)
next capacity 1280*1.25 = 1600, but i see 1696 (len 1281).


Why difference = 96?


len 1281 - 3073 wrong, but len 3073 * 1.25 = 5120 (len 4097)

And if golang can ramp up capacity array when increase slice, can it reduce the array when the slice that refers to it is too small?

Thank you!

The only one who knows the requirements of when a slice needs to be smaller is you, and it’s very easy to set a new capacity or make a new slice. — JimB, Jan 10 '20 at 11:57
How slices grow is not specified. The Go compiler might do different things on Monday than on Tuesday. There is nothing to know or validate here, (almost) any behavior is compliant with the spec. — Volker, Jan 10 '20 at 13:20
@Volker: How slices grow is specified, it is specified by the implementation, the current implementation in the code that the OP linked to. The OP is asking why that code doesn't produce the results they expected. — peterSO, Jan 10 '20 at 14:47
@peterSO Contrary to lots of other languages Go is not specified by the implementation but by the language specification. — Volker, Jan 10 '20 at 15:10
@Volker: The specification refers to an implementation. For example, "There is also a set of predeclared numeric types with implementation-specific sizes". I don't share your opinion. — peterSO, Jan 10 '20 at 16:28
The specification refers to what implementations can do, it is not supplemented by them. This capacity question is a perfect example of the difference between specification and implementation. This particular implementation can, and has changed multiple times, all while remaining within specification. The gccgo implementation works slightly differently, but also still confirms to specification. — JimB, Jan 11 '20 at 13:30

peterSO · Accepted Answer · 2020-01-13T20:36:25.910

next capacity 1280*1.25 = 1600, but i see 1696 (len 1281).

src/runtime/malloc.go

// Small allocation sizes (up to and including 32 kB) are
// rounded to one of about 70 size classes, each of which
// has its own free set of objects of exactly that size.

growslice requests an minimum allocation size. mallocgc performs the allocation. mallocgcrounds up to a class size to minimize fragmentation.

Write a shrink memory function.

package main

import "fmt"

func shrink(s []byte) []byte {
    if (cap(s)-len(s))/len(s) >= 1 {
        t := s
        s = make([]byte, len(s), 2*len(s))
        copy(s, t)
    }
    return s
}

func main() {
    s := make([]byte, 32, 256)
    fmt.Println(len(s), cap(s))
    s = shrink(s)
    fmt.Println(len(s), cap(s))
}

Playground: https://play.golang.org/p/udAEBJ-kWQ9

Output:

32 256
32 64

As you can see it costs time and memory. Only you can decide whether it is worth it in your particular case.

Extend memory when grow slice (logic grow capacity underlying array)

1 Answers1