How do I set audit related flags on kubeapi-server when using kubeadm?

Question

I'm trying to enable 'auditing'. https://kubernetes.io/docs/tasks/debug-application-cluster/audit/ mentions:

You can pass a file with the policy to kube-apiserver using the --audit-policy-file flag. If the flag is omitted, no events are logged.

I've used kubeadm to configure the cluster (running in 3 VMs in total).

However where is this set when using kubeadm ? I don't see where it interacts with kube-apiserver.

kubelet and the apiserver are completely different. How did you provision your cluster? Kops? Kubeadn? Kubespray? — jaxxstorm, Jul 30 '18 at 21:38
I used kubeadm. IIRC I start the 'kubernetes' service via kublet. — Chris Stryczynski, Jul 30 '18 at 21:45

score 5 · Answer 1 · answered Mar 31 '20 at 09:08

For a recent version of Kubernetes, add this to the kind: ClusterConfiguration section:

    apiServer:
      extraArgs:
        audit-log-path: /tmp/k8s-audit.log
        audit-policy-file: /etc/kubernetes/audit.yaml
        audit-log-maxage: "1"
        audit-log-maxsize: "100"
        audit-log-maxbackup: "1"
      extraVolumes:
        - name: audit
          hostPath: /host/audit.yaml
          mountPath: /etc/kubernetes/audit.yaml
          readOnly: true
          pathType: File

Example /host/audit.yaml file which logs all request and response bodies:

apiVersion: audit.k8s.io/v1
kind: Policy
omitStages:
  - "ResponseStarted"
rules:
  - level: RequestResponse

An extra volume for the files might work as well, with the logs on that instead of /tmp — Gert van den Berg, May 08 '20 at 14:57

score 4 · Answer 2 · answered Jul 30 '18 at 22:15

Kubelet and kube-apiserver are entirely different components of the Kubernetes stack.

The apiserver is part of the Kubernetes Controlplane. The Kubelet itself actually connects to the API Server. What you're getting confused about is the fact that generally, the API Server runs as a docker container, and this docker container can be managed by the kubelet.

The kubelet can run containers in two ways:

Decided by the control plane (request sent by the scheduler)
As a static manifest in the form of yaml or JSON under a directory (default /etc/kubernetes/manifests

If you're using kubeadm, take a look under this directory. You'll see a yaml file with a manifest for the API server. You can add your apiserver flag to this manifest and it'll restart the API server with the new flag.

However, as stated in the comments, you're using kubeadm. There's a correct way to do this which will persist after upgrades. If you look at the documentation page for kubeadm init you'll see there's an option "Using kubeadm init with a configuration file"

So you'd write a yaml configuration file for kubeadm which would look a bit like so in config.yaml:

apiVersion: kubeadm.k8s.io/v1alpha2
auditPolicy:
  logDir: /var/log/kubernetes/audit
  logMaxAge: 2
  path: ""

Then run kubeadm init --config config.yaml

Appreciate the answer, it's informative and well written! Is it necessary to run `kubeadm init`? Or is it possible to update the config on an existing cluster? I've tried `kubeadm config upload from-file --config config.yaml` with no luck. — Chris Stryczynski, Jul 31 '18 at 17:02
This does not seem to work? `/var/log/kubernetes/audit` remains empty even with `777` permissions. — Chris Stryczynski, Aug 01 '18 at 12:24

Chris Stryczynski · Accepted Answer · 2019-02-17T22:18:02.080

apiVersion: kubeadm.k8s.io/v1alpha2
api:
  advertiseAddress: 192.168.0.33
  bindPort: 6443
networking:
  podSubnet: "10.244.0.0/16"
featureGates:
  Auditing: true
auditPolicy:
  logDir: "/var/log/kubernetes/"
  logMaxAge: 20
  path: "/etc/kubernetes/audit.yaml"

And

cat /etc/kubernetes/audit.yaml

:

# Log all requests at the Metadata level.
apiVersion: audit.k8s.io/v1beta1
kind: Policy
rules:
- level: Metadata

Found the answer here: https://evalle.xyz/posts/how-to-enable-kubernetes-auditing-with-kubeadm/.

How do I set audit related flags on kubeapi-server when using kubeadm?

3 Answers3