What a CNI Plugin Chooses
What a CNI Plugin Chooses
Kubernetes requires the three networking rules but not how to meet them; the CNI plugin decides the datapath:
- Encapsulation — VXLAN wraps pod traffic in UDP (simple, works anywhere) versus BGP or cloud routes that carry native pod IPs (no overlay overhead, more network integration).
- IPAM — who hands out pod IPs: host-local blocks, Calico's IP pools, or cloud allocations.
- Policy enforcement — iptables/eBPF rules translating NetworkPolicy into drops.
- Extras — BGP peering with routers, bandwidth shaping, dual-stack, Windows nodes.
ls /etc/cni/net.d # configured CNI (e.g. 10-calico.conflist)
kubectl get pods -n kube-system -l k8s-app=calico-node
The kubelet invokes whatever /etc/cni/net.d declares each time a pod sandbox is created.
Calico, Cilium, Flannel
Calico, Cilium, Flannel
- Flannel — the minimal option: VXLAN (or host-gw) networking with per-node pod CIDRs. Fast to deploy; NetworkPolicy enforcement is not its strength, so pair it elsewhere or skip policies.
- Calico — networking plus a full policy engine. Supports VXLAN or pure routed BGP (peering with top-of-rack switches), rich policies beyond the Kubernetes API (globals, tiers), and works on bare metal and cloud alike.
- Cilium — eBPF-based: replaces kube-proxy's iptables with BPF programs, adds L3/L4/L7 policy, transparent encryption, and Hubble for flow observability. The cost is kernel requirements (recent ≥5.x for full features) and operational learning curve.
For throughput-sensitive clusters, eliminating iptables resync storms with Cilium or IPVS is a real win.
Pitfalls and Interview Angles
Pitfalls and Interview Angles
Pitfall 1 — swapping CNIs on a live cluster. Pods keep old routes and IPs; a CNI migration drains and re-creates workloads (or rebuilds the cluster). Also remove stale /etc/cni/net.d entries so kubelet stops invoking the old plugin.
Pitfall 2 — believing policies on Flannel. Objects apply without error and traffic stays open — a dangerous false sense of security. Verify with an actual probe, not kubectl get netpol.
Interview-ready: contrast overlay (VXLAN) versus routed (BGP) with a concrete tradeoff — encapsulation cost versus network integration — name who runs in your cluster by inspecting /etc/cni/net.d, and explain how eBPF lets Cilium attach policy at socket level instead of per-packet iptables.
Quiz
1. Where does kubelet learn which CNI plugin to run?
2. What is the main advantage of BGP-routed pod networking over VXLAN?
3. Which CNI is known for eBPF datapath and Hubble observability?
Flashcards
Question
CNI plugin responsibilities
Click to reveal answer
Answer
Attach interfaces, assign IPs, install routes/tunnels, and often enforce policy.
Question
VXLAN vs BGP
Click to reveal answer
Answer
VXLAN = portable overlay with encapsulation cost; BGP = native pod IPs, needs fabric integration.
Question
Flannel's profile
Click to reveal answer
Answer
Minimal VXLAN/host-gw networking; not the tool for full NetworkPolicy enforcement.
Question
Cilium's differentiator
Click to reveal answer
Answer
eBPF replaces kube-proxy iptables, adds L7 policy, encryption, and Hubble flow observability.
Revision Notes
Key Takeaways
- 1. The CNI decides encapsulation, IPAM, and policy datapath
- 2. CNI migration on live clusters is a drain-and-recreate operation
- 3. Verify policies with probes, not with object existence
Interview Tips
- • Name the CNI in your environment and justify it with one concrete requirement
- • Explain overlay versus routed with the encapsulation/MTU tradeoff
Cheat Sheet
CNI from /etc/cni/net.d. VXLAN = overlay cost; BGP = native IPs. Flannel minimal, Calico policy+BGP, Cilium eBPF+Hubble. CNI swaps need drain/recreate.