Setting POSIX message queues RLIMIT_MSGQUEUE to unlimited from Go for a Kubernetes container

Question

We have a library that uses POSIX message queues to pass data between processes and requires having the msgqueue kernel parameter set to unlimited, otherwise the process will return an error too many open files. Usually we view/set this using ulimit -q and ulimit -q unlimited, but for some Kubernetes containers we want to be able to set this directly via go code.

We have already confirmed that the pod spec allows changing mqueue settings (via securityContext for Kubernetes 1.14+) and that the specific container can adjust resource setting:

...
kind: Deployment
...
spec:
  template:
    spec:
      securityContext:
        sysctls:
        - name: fs.mqueue.msg_max
          value: "10000"
        - name: fs.mqueue.msgsize_max
          value: "102400"
      containers:
      - name: testing
        securityContext:
          capabilities:
            add:
            - SYS_RESOURCE
...

Without those settings even ulimit -q unlimited would get an error:

bash: ulimit: POSIX message queues: cannot modify limit: Operation not permitted

But how would we use syscall.Setrlimit to adjust the limit value from our code?

score 1 · Answer 1 · answered May 20 '20 at 19:10

Looks like the syscall package was frozen without the RLIMIT_MSGQUEUE = 0xC constant, and it also defines syscall.RLIM_INFINITY = -0x1 which results in an error when attempting to use that value:

syscall: Rlimit constant -1 overflows uint64

So instead you have to manually define the constants

const RLIMIT_MSGQUEUE int = 0xC
const RLIM_INFINITY uint64 = 0xffffffffffffffff // https://github.com/golang/go/issues/22731
err = syscall.Setrlimit(RLIMIT_MSGQUEUE, &syscall.Rlimit{RLIM_INFINITY,RLIM_INFINITY}))

or switch to using the methods from https://godoc.org/golang.org/x/sys/unix:

err = unix.Setrlimit(unix.RLIMIT_MSGQUEUE, &unix.Rlimit{Cur: unix.RLIM_INFINITY, Max: unix.RLIM_INFINITY})

also note that the limit values are set only for that process and it's children, so to confirm you have to either check /proc/<pid>/limits or call Getrlimit/ulimit from the code as well:

var rLimit syscall.Rlimit
err := syscall.Getrlimit(0xC, &rLimit)
if err != nil {
    fmt.Println("Error Getting Rlimit ", err)
}
fmt.Println(rLimit)

// construct bash command
cmd := &exec.Cmd {
    Path: "/bin/bash",
    Args: []string{ "/bin/bash", "-c", "ulimit -q"},
    Stdout: os.Stdout,
    Stderr: os.Stdout,
}

// run `cmd` in background and wait for it to finish
cmd.Start()
cmd.Wait()

{18446744073709551615 18446744073709551615}

unlimited

Setting POSIX message queues RLIMIT_MSGQUEUE to unlimited from Go for a Kubernetes container

1 Answers1