I am currently running into an issue where /sys/fs/cgroup/cpuset.cpus is no longer present in my Kubernetes Pod when I enable Privileged mode.
For example:
I start off with a very simple pod spec (test.yaml
):
apiVersion: v1
kind: Pod
metadata:
name: test-0
spec:
nodeSelector:
kubernetes.io/hostname: "node1"
containers:
- name: pod-0
image: ubuntu:22.04
command: [ "/bin/sh" , "-c", "tail -f /dev/null" ]
resources:
requests:
cpu: 4
memory: 2Gi
limits:
cpu: 4
memory: 2Gi
I start the pod and exec into it:
kubectl apply -f test.yaml
kubectl exec -it test-0 -- /bin/bash
Once in the container, I run the following:
cat /sys/fs/cgroup/cpuset.cpus
The output is 44,46,100,102
i.e the cpus allocated to my pod are printed as expected using cgroupv2 filesystem
Next I remove the pod (kubectl delete pod test-0
) and create a new pod with the following spec (note the new privileged securityContext):
apiVersion: v1
kind: Pod
metadata:
name: test-0
spec:
nodeSelector:
kubernetes.io/hostname: "node1"
containers:
- name: pod-0
image: ubuntu:22.04
securityContext:
allowPrivilegeEscalation: true
privileged: true
command: [ "/bin/sh" , "-c", "tail -f /dev/null" ]
resources:
requests:
cpu: 4
memory: 2Gi
limits:
cpu: 4
memory: 2Gi
Same as before, I start the pod and exec into it:
kubectl apply -f test.yaml
kubectl exec -it test-0 -- /bin/bash
Again, once in the container, I run the following:
cat /sys/fs/cgroup/cpuset.cpus
But this time I get the following error:
cat: /sys/fs/cgroup/cpuset.cpus: No such file or directory
Why is this happening whereby adding privileges to my container, it removes the /sys/fs/cgroup/cpuset.cpus file from my container in Kubernetes?
Kubernetes Version: 1.26.2
Cgroup driver: systemd
Containerd Version: 1.6.20
OS Version: Ubuntu 22.04.2
Kernel: 5.15.0-76-generic