Pipeline Stages
Pipeline Stages
A Kubernetes pipeline moves one commit through:
- Verify — lint, unit tests (fail fast, minutes not hours)
- Build — produce an immutable image; in-cluster jobs use kaniko (unprivileged, no daemon) or Buildpacks/BuildKit, since Docker-in-Docker needs privileged pods
- Scan & sign — Trivy/Grype for CVEs, record the digest (
registry/app@sha256:...), generate an SBOM, sign with cosign - Deliver — open a commit updating the Git manifest with that digest (or tag), not apply directly
- Verify — after the CD agent syncs, run
kubectl rollout status/smoke tests, then gate promotion to the next environment
CI owns building and evidence; CD owns deployment state — separation that keeps credentials narrow and history clean.
Wiring CI to the Cluster
Wiring CI to the Cluster
GitOps path (preferred): the pipeline never holds kubeconfig — it only pushes to Git:
yq -i ".spec.template.spec.containers[0].image = \"$REG/app@sha256:$DIGEST\"" \
deploy/base/deployment.yaml
git commit -am "deploy: app $DIGEST"; git push
Argo CD/Flux picks up the commit and syncs; the audit trail is the PR.
Push path (simpler, weaker): CI applies directly:
kubectl -n prod set image deploy/app app=$REG/app:$TAG
kubectl -n prod rollout status deploy/app --timeout=180s
This needs a kubeconfig secret in CI, works for single-cluster side projects, and loses in-cluster drift correction — know which trade-off you are making.
Pitfalls and Interview Angles
Pitfalls and Interview Angles
Mutable tags. latest or rebuilt tags make "what ran in prod?" unanswerable — pin digests and treat images as write-once.
Secrets in logs. Mask CI variables, avoid set -x around credential steps, and use short-lived OIDC-federated tokens instead of long-lived kubeconfigs where your CI supports it.
No gates. Deploying on green unit tests alone ships untested integration behavior — add e2e and a rollout-status/health check before declaring success.
Privileged builders. DinD privileged pods widen the attack surface; kaniko/BuildKit rootless builds avoid it.
Interview angle: narrate a commit to production end-to-end (tests → build → scan → digest commit → sync → verify), naming where the audit trail lives at each step.
Quiz
1. Why prefer kaniko over docker-in-docker in CI pods?
2. In a GitOps pipeline, what does CI write when a new image is ready?
3. Why pin images by digest instead of tag?
Flashcards
Question
CI vs CD ownership
Click to reveal answer
Answer
CI builds and attests artifacts; CD (GitOps agent) converges clusters to Git — separate credentials and histories.
Question
kaniko
Click to reveal answer
Answer
Rootless, daemonless image builder for unprivileged CI pods — the DinD alternative.
Question
rollout status gate
Click to reveal answer
Answer
kubectl rollout status deploy/x --timeout=... verifies the deploy actually became ready before promotion.
Question
OIDC in CI
Click to reveal answer
Answer
Short-lived federated tokens for cloud/cluster auth instead of stored kubeconfig secrets.
Revision Notes
Key Takeaways
- 1. Stages: verify → build → scan/sign → commit digest → CD syncs → verify again
- 2. Prefer GitOps hand-off so CI never holds cluster credentials
- 3. Digests, SBOMs, and signatures make "what shipped" a fact, not a guess
Interview Tips
- • Describe a commit-to-production path and point to the audit artifact at each stage
- • Defend digest pinning and explain what breaks when tags are reused
Cheat Sheet
verify → build (kaniko/buildpacks) → scan/sign → commit digest to Git → CD syncs → rollout status gate. CI holds no kubeconfig in GitOps; images write-once by digest; mask secrets, prefer OIDC.