Security Auditing

Evidence Before Controls

Auditing answers who did what and whether controls held. Two complementary tool families:

Kubernetes audit logs record every API request with user, verb, resource, source IP, and optionally bodies — shaped by an audit policy:

rules:
- level: RequestResponse
  resources:
  - group: ""
    resources: ["secrets"]
- level: Metadata
- level: None
  users: ["system:kube-proxy"]

The API server writes to files or a webhook backend; omitStages trims noisy stages like RequestReceived.

kube-bench checks node and control-plane configuration — file permissions, flag settings, RBAC defaults — against the CIS Kubernetes Benchmark and reports pass/fail per control.

Operating an Audit Setup

kube-bench run --targets master,node
kubectl get --raw /apis | jq '.groups[].name'

Practical rules:

• Ship logs off-box — webhook backend or a log agent, so node compromise cannot erase the trail.
• Keep levels honest — Metadata minimum everywhere, RequestResponse where bodies matter (secrets, RBAC changes), None for kubelet watch floods.
• Cover the node plane too — audit logs capture API actions, not container syscalls; add a runtime tool like Falco for the latter.

Review loop: alert on privileged pod creation and RBAC binding changes, sample weekly, and convert findings into admission rules that prevent recurrence.

Pitfalls and Interview Angles

Pitfall 1 — local-only logs with short rotation. Files that rotate in hours help nobody during a post-incident review. Centralize with immutability and agree retention measured in months where compliance demands it.

Pitfall 2 — trusting a past kube-bench run. New releases change defaults and flags; rerun after every cluster upgrade and after any control-plane configuration change.

Interview-ready: separate coverage domains — audit logs for the API plane, CIS/kube-bench for configuration, runtime tools for container behavior — name the first policy rule you would ship (RequestResponse on secrets and role bindings), and explain how findings flow back into admission policy so the same violation cannot repeat.