`docker system df` vs `/system/df` (docker api endpoint)

Question

I am writing a program in Go to get the total disk usage in GB from my docker host machine. For that, I am using the func DiskUsage() from go lib:

• https://pkg.go.dev/github.com/docker/docker/client#Client.DiskUsage.

Taking a look into the code, this func is calling the docker api endpoint /system/df:

• https://docs.docker.com/engine/api/v1.43/#tag/System/operation/SystemDataUsage

However, I noticed a weird behaviour when I calculate the GB using this lib vs using the command docker system df:

• docker system df output:

  $ docker system df
  TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
  Images          223       4         21.02GB   20.7GB (98%)
  Containers      6         0         0B        0B
  Local Volumes   13        1         536.4MB   340.4MB (63%)
  Build Cache     954       0         13.51GB   13.51GB
  

• My Go application output:

  $ go run ./cmd/main.go
  Images: TOTAL (223), 17GB
  Build Cache: TOTAL (954), 29GB
  

As you can see, we have a difference between both outputs. I would like a help to understand if I am doing something wrong with this calculation which gets data from the /system/df endpoint. Thanks :)

Go application:

package main

import (
    "context"
    "fmt"

    "github.com/docker/docker/api/types"
    "github.com/docker/docker/client"
)

func main() {
    cli, err := client.NewClientWithOpts(client.FromEnv)
    if err != nil {
        panic(err)
    }

    diskUsg, err := cli.DiskUsage(context.Background(), types.DiskUsageOptions{})
    if err != nil {
        panic(err)
    }
    b := float64(0)
    for _, ch := range diskUsg.BuildCache {
        b = b + float64(ch.Size)
    }

    b2 := float64(0)
    for _, ch := range diskUsg.Images {
        if ch.Size > ch.SharedSize {
            b2 = b2 + (float64(ch.Size) - float64(ch.SharedSize))
            continue
        }
        b2 = b2 + (float64(ch.SharedSize) - float64(ch.Size))
    }

    fmt.Printf("Images: TOTAL (%d), %2.fGB\n", len(diskUsg.Images), float64(b2)/(1<<30))
    fmt.Printf("Build Cache: TOTAL (%d), %2.fGB\n", len(diskUsg.BuildCache), float64(b)/(1<<30))
}

Answer 1

Based on Docker source code:

• system df command: https://github.com/docker/cli/blob/v24.0.5/cli/command/system/df.go
• output formatting: https://github.com/docker/cli/blob/v24.0.5/cli/command/formatter/disk_usage.go

You should be able to reproduce exactly what docker system df does using the code below:

• go.mod

module 76982562-docker-system-df-vs-system-df-docker-api-endpoint

go 1.21.0

require (
    github.com/docker/cli v24.0.5+incompatible
    github.com/docker/docker v24.0.5+incompatible
)

• main.go

package main

import (
    "context"
    "fmt"
    "os"

    "github.com/docker/cli/cli/command/formatter"
    "github.com/docker/docker/api/types"
    "github.com/docker/docker/client"
    "github.com/docker/go-units"
)

func main() {
    cli, err := client.NewClientWithOpts(client.FromEnv)
    if err != nil {
        panic(err)
    }

    diskUsage, err := cli.DiskUsage(context.Background(), types.DiskUsageOptions{})
    if err != nil {
        panic(err)
    }

    var bsz int64
    for _, bc := range diskUsage.BuildCache {
        if !bc.Shared {
            bsz += bc.Size
        }
    }

    fmt.Printf("Images: TOTAL (%d), %s\n", len(diskUsage.Images), units.HumanSize(float64(diskUsage.LayersSize)))
    fmt.Printf("Build Cache: TOTAL (%d), %s\n", len(diskUsage.BuildCache), units.HumanSize(float64(bsz)))
}

• for images, docker library provides directly diskUsage.LayersSize to represent the total size, so you don't have to compute it yourself
• for build cache, you need to exclude shared items (if !bc.Shared)

To convert sizes in the right unit, I highly recommend to use github.com/docker/go-units (e.g. units.HumanSize(float64(diskUsage.LayersSize))). This will avoid you unit conversion nightmares!