CAP and PACELC Trade-offs

At 02:14 the switch between your two availability zones drops half the traffic for eleven minutes. Nothing else is wrong — every process runs, every disk is healthy. During those eleven minutes, what does your key-value store return: a value, or an error? CAP says you must decide now, because during a partition consistency (every read sees the latest completed write) and availability (every non-failing node answers) cannot both be held.

The argument is small enough to derive in an interview:

Client C writes x=1 to node N1. The N1↔N2 link is partitioned.
N2 still receives reads. Two choices for N2's answer to read(x):

  answer x=0 (stale)  → AP: available, but C's completed write is invisible
                         ⇒ not linearizable consistency
  refuse/error        → C: consistent with x=1, but N2 did not serve
                         ⇒ not availability
There is no third answer: N2 cannot learn the new value because
the very link that would carry it is the one that is broken.

Real systems place themselves deliberately on this line:

Two clarifications interviewers probe for. First, CP often means 'partial unavailability by design' — ZooKeeper's minority serving no writes is the correct, safety-preserving answer, not a bug; and a system can be CP for some operations and AP for others (Kafka is CP for producer acks with acks=all and min-ISR unmet, but consumers reading from an out-of-date replica are being served stale data on purpose). Second, availability in CAP is defined narrowly: a node that responds to every request with HTTP 503 is available by liveness but useless to you; the formal definition means a non-failing node returns a response — so CAP's 'A' is about responding, not about responding usefully, which is why the debate gets muddled.

Also worth saying: partitions are not hypothetical and not only cable cuts. A GC pause of 800 ms, a full disk queue, a confused load balancer, or an asymmetric routing failure all present as partitions to the protocols — timeouts are how a partition is detected, and every partition-time decision is therefore a decision you make under uncertainty about whether the other side is dead or just slow. That uncertainty is exactly what consensus resolves with terms and majorities (Consensus Overview with Raft), and what replication mode determines in the first place (Replication Fundamentals).

The practical move: do not pick a side for the whole database. Classify operations. reserveSeat(seatId) must refuse on the minority side (C) or you oversell; recordPageView must keep accepting (A) or the analytics pipeline stops for eleven minutes for no benefit. Most mature systems expose this per operation: Kafka's acks and enable.idempotence, Cassandra's per-query CONSISTENCY, DynamoDB's on-demand vs strongly consistent reads. Answering with a per-operation classification is how you pass this question.

Your cluster ran for a year with 4 minutes of partition. For the other 525,596 minutes, CAP had nothing to say — yet you still chose synchronous replication over asynchronous, quorum reads over local reads, writeConcern=majority over writeConcern=1. Those choices trade latency for consistency while everything is healthy, and CAP does not model them at all. PACELC (Abadi, 2012) adds the missing half: if there is a Partition, choose between Availability and Consistency; Else, choose between Latency and Consistency.

That framing produces four labeled quadrants, which is why it is more useful than a CP/AP badge:

                 partitioned          healthy
        ┌──────────────────────┬──────────────────────┐
 choose │  P-A : Cassandra,    │  E-L : DynamoDB default,
        │  DynamoDB on-demand, │      MongoDB w:1 j:false,
        │  Riak               │      Cassandra LOCAL_ONE
        ├──────────────────────┼──────────────────────┤
        │  P-C : ZooKeeper,    │  E-C : Spanner,
        │  etcd, Kafka acks=all│      Postgres sync-commit,
        │  (minority refuses)  │      MongoDB w:majority
        └──────────────────────┴──────────────────────┘

Reading the bottom-right row is the point: Spanner and a w=majority Postgres are both E-C — they pay RTT on every operation during normal operation, which is why cross-region synchronous replication shows up as p99 latency before it ever shows up as an outage. The top-left row prices the opposite call: Cassandra tuned for LOCAL_ONE accepts staleness in the healthy case (E-L) and accepts divergence during partitions (P-A) — one tuning philosophy expressed in two regimes.

Concrete numbers make this interviewable:

The design discipline PACELC forces: write down both answers. A system description that says only 'we are AP' tells you nothing about whether it adds 1 ms or 150 ms to your write path in production — and the E-C/E-L decision is the one your users feel every day, while the P-C/P-A decision is the one your incident review feels once a year.

A useful exercise is to score your own service on all four cells. Take an order API: with writeConcern=majority on the primary and reads at readConcern=linearizable for checkout, it is E-C; if the primary's region is cut off, orders stop — P-C. Now the recommendation feed: reads at readConcern=available from secondaries and writes acked at w:1 — E-L for both halves, so a partition means some users see older suggestions for a while. Two collections inside one cluster, two PACELC answers, one per operation rather than one per vendor. Write the table down in the design doc; it turns arguments about 'strong vs eventual' into a list of settings someone can review and change.

One trap: PACELC still treats the system as one point, and real deployments are per-operation hybrids — Kafka is E-C for producers with acks=all and E-L for consumers reading followers; MongoDB can be w:majority for the ledger and w:1 for the session store in the same process. The labels are a starting vocabulary, not the conclusion; the next chapter is about why the conclusion needs a scenario attached.

Ask three engineers whether their system is CP or AP and you will get three answers, all defensible — because the label is meaningless until you name which partition you are talking about. CAP constrains behavior only for the specific partitioned scenario assumed, and systems have several:

• One node is isolated from its peers (the classic majority/minority split). ZooKeeper is CP here: the minority refuses writes.
• One node is isolated from the clients but connected to peers. Now that node serves nothing while the cluster is perfectly consistent and available — no C/A choice was made at all.
• A client is partitioned from part of the cluster (a load balancer or firewall fault). Reads from stale replicas may be served while the cluster itself is healthy — AP-looking behavior from a CP system.
• A node is slow, not dead — a 700 ms GC pause. Protocols must treat it as a partition (timeouts cannot distinguish), so the choice happens even though the network is fine.

Say that list in an interview and the question changes shape: instead of 'we are CP', the senior answer is 'when node-to-node links partition, our primary and secondary coordination tier fails closed — minority refuses writes — while our read path degrades to stale-but-serving after the lease expires, so per operation we are CP for writes and AP for reads.' That sentence contains verifiable behavior; a badge does not.

The framework that produces it:

For each operation:
 1. Name the invariant it protects.        (no double-sell, no lost transfer,
                                             page views may drop)
 2. Name the partition scenario assumed.    (who can be cut off from whom,
                                             and for how long before failover)
 3. State the answer for both sides.        (minority refuses / minority serves stale,
                                             with the reconciliation path named)
 4. State the healthy-case latency budget.  (the E half of PACELC)
 5. Name the reconciliation mechanism.      (repair, anti-entropy, saga, leader election)

Step 5 is the part most candidates skip, and it is what makes an AP choice responsible rather than careless: an AP system is not 'eventually fine', it is 'divergent and equipped with a merge function, versioning, and a repair path' (Consistency Models covers those mechanisms). A CP system's counterpart is an election and fencing story. Every CAP answer is really a pair: a refusal policy and a reconciliation policy. Walk a real example through the five steps and the framework shows its value: a shopping cart stored with per-write versioning, AP during a partition because abandoned carts tolerate merged items, reconciled by union-with-tombstones on healing — while the inventory reservation in the same request path is CP, refusing reservations on the minority side and reconciling through the saga's compensating release. Same outage, opposite answers, both correct.

A final practical note: CAP-shaped decisions are usually made by a handful of configuration values you can point at — min.insync.replicas, writeConcern, ConsistencyLevel, lease TTLs, and timeout thresholds — so an architecture review can audit them. Ask which of those settings implements the claimed badge; if nobody can name one, the label is folklore. Those same timeout and quorum parameters decide how your system behaves when messages pile up, which is where the roadmap goes next: Message Queues and Brokers.