K8S Auditing - how to get all the policy options?

Question

I would like to define policy for audit log that will include information only regarding pod creation. I would like to make sure that all the audit logs wont appear. how can i find all the options that the policy can have? I would like to have only

- level: RequestResponse
    resources:
    - group: ""
      # Resource "pods" doesn't match requests to any subresource of pods,
      # which is consistent with the RBAC policy.
      resources: ["pods"]

I used k8s formal site and use their example, I changed everything to "None" except of the "Pods" rule, and still i got a lot of other logs not related to pods.

my policy is:

omitStages:
  - "RequestReceived"
rules:
  # Log pod changes at RequestResponse level
  - level: RequestResponse
    resources:
    - group: ""
      resources: ["pods"]

  - level: None
    resources:
    - group: ""
      resources: ["pods/log", "pods/status"]

  - level: None
    resources:
    - group: ""
      resources: ["configmaps"]
      resourceNames: ["controller-leader"]

  - level: None
    users: ["system:kube-proxy"]
    verbs: ["watch"]
    resources:
    - group: "" # core API group
      resources: ["endpoints", "services"]

  - level: None
    userGroups: ["system:authenticated"]
    nonResourceURLs:
    - "/api*" # Wildcard matching.
    - "/version"

  # Log the request body of configmap changes in kube-system.
  - level: None
    resources:
    - group: "" # core API group
      resources: ["configmaps"]

  - level: None
    resources:
    - group: "" # core API group
      resources: ["secrets", "configmaps"]

  - level: None
    resources:
    - group: "" # core API group
    - group: "extensions" # Version of group should NOT be included.


  - level: None
    # Long-running requests like watches that fall under this rule will not
    # generate an audit event in RequestReceived.
    omitStages:
      - "RequestReceived"```

i use this flag (--audit-policy-file) in order to set my policy. the problem is that the auditing generate a lot of logs that i dont need - i would like to get logs only on pod creation and that it. is there any way to understand how can i block all the rest? — inza, Jan 09 '20 at 05:40
i added my policy in my original question. thanks for your assistance — inza, Jan 12 '20 at 08:40
i got these: ResponseComplete","verb":"patch","user":{"username":"system:node:kubernetes-minion","groups":["system:nodes","system:authenticated"]}, "objectRef":{"resource":"pods","subresource":"status"}, "ResponseComplete","verb":"patch","user":{"username":"system:node:kubernetes-minion","groups":["system:nodes","system:authenticated"]}, bjectRef":{"resource":"nodes","subresource":"status"}, i added this rule:- level: None userGroups: ["system:nodes","system:authenticated"] verbs: ["patch"] resources: - group: "" resources: ["nodes/status","pods/status"] but still got them-why? — inza, Jan 16 '20 at 06:39

Wytrzymały Wiktor · Answer 1 · 2020-01-14T10:28:33.260

Simply changing the example config .yaml will not solve your issue. You need to understand exactly how the audit policy file works.

When an event is processed it is compared against the audit policy rules in order, and the first matching rule sets the audit level of the event.

Here's an example of a rule you want to setup in your use case. If an event matches the rule, the Kubernetes API server creates a log entry at the RequestResponse level.

- level: RequestResponse
  verbs: ["create"]
  resources:
    - group: "" # core
      resources: ["pods", "pods/status"]
  omitStages:
    - "RequestReceived"

An event matches the rule if all of the following are true:

The event does not match any previous rule in the policy file.
The call is a create request.
The request is on a pods resource or a pods/status resource.
The event is not for the RequestReceived stage of the call.

If you still log entries that you would like to filter out than add level: None rules before the above example, as the Kubernetes audit policy file starts with rules that specify that certain events should not be logged at all.

Please let me know if that helps.

EDIT:

how can i make sure that i cover all K8S audit options? is there any "general" rule that say dont log anything and before this rule i will add only my specific logs?

It depends on your cluster. The easiest way to achieve that would be to:

set the rule you want to log
check the logs if they catch anything you don't want
add a level: None rule before the one you created to exclude the unnecessary events

regarding the pod creation if i would like to be more specific and get logs only from "default" namespace and not form other namespaces

The same thing can be applied here. Just add a level: None rule that would exclude events from other namespaces.

I added the following policy and changed all to none except the pod creation that i added in the end:https://github.com/kubernetes/kubernetes/blob/release-1.10/cluster/gce/gci/configure-helper.sh#L706 and i still have some questions: * how can i make sure that i cover all K8S audit options? is there any "general" rule that say dont log anything and before this rule i will add only my specific logs? — inza, Jan 14 '20 at 08:53
regarding the pod creation if i would like to be more specific and get logs only from "default" namespace and not form other namespaces i wrote the following: ``` - level: RequestResponse namespaces: ["default"] verbs: ["create"] resources: - group: "" # core resources: ["pods", "pods/status"] - level: None verbs: ["create"] resources: - group: "" # core resources: ["pods", "pods/status"] ``` but i got log for any pod creation , not only on default namespace. what am i missing? — inza, Jan 14 '20 at 08:54

K8S Auditing - how to get all the policy options?

1 Answers1