Graceful Degradation and Backpressure

Your homepage is assembled from twelve backend calls: catalog, pricing, inventory, reviews, recommendations, ads, geolocation, A/B assignments. At 14:07 the recommendations service starts returning after 4 seconds. You have two options. Fail the whole page because one optional widget is late — every customer sees a blank screen while a working catalog sits idle behind it. Or render the page without recommendations and keep the money path alive. The second option is only available if someone already decided, calmly and before the incident, which features may be dropped and what replaces them.

That decision artifact is the degradation ladder: for each feature, its business criticality, its degraded mode, and the exact condition that triggers the downgrade.

The table forces the two conversations that actually matter. First: some things must not degrade — authentication, payment correctness, data integrity. Graceful degradation is choosing what to stop doing, never a license to do it wrong; a checkout that skips fraud checks during load converts an availability problem into a financial one, so the fraud row is a deliberate risk acceptance with a dollar ceiling attached to it. Second: everything optional needs a precomputed answer. A fallback designed during the incident is a fallback that does not exist.

The mechanics of each degraded mode differ in an important way. Hiding a widget is a boolean. Serving stale data is a contract: bound it with a TTL, label it (show the 15-minute-old price as such, or don't show it), and only ever on read paths — staleness is acceptable for display, never for a write decision. Static shells work when the failure is client-side or edge-side: a CDN-cached HTML skeleton that boots without the personalization API still gives a customer something to read. Read-only mode is the classic dependency-down posture for write-heavy systems: keep serving the data you already have, reject or queue mutations. And when the synchronous path genuinely cannot be honored — but the work itself is valuable — return 202 Accepted with a job reference and finish asynchronously; that converts a failure into latency, provided you can show honest progress.

Two patterns make the ladder operable rather than aspirational. Feature flags (or kill switches) let you flip to a degraded mode in seconds without a deploy, and they are the same mechanism you will toggle during the game day in the final topic. Breaker-driven degradation automates it: when the recommendations breaker opens, the composition layer stops calling it and renders the fallback, with no human in the loop at 3 a.m.

The trade-off you are accepting is explicit: degraded responses cost extra code paths, extra tests, and extra metrics per feature, and every flag is one more configuration surface that can be wrong. The alternative is worse — all-or-nothing availability means one optional dependency takes the entire product down, and your availability number becomes the product of every dependency's uptime instead of just the ones that matter. Bulkheads and rate limits from Bulkheads and Rate Limiting decide how much work runs; the ladder decides which work deserves the capacity you have left.

Interviewers ask "how do you decide what to degrade?" Answer with the artifact, not adjectives: a per-feature criticality ranking tied to business impact, a named degraded mode per feature, a measurable trigger, and an owner — all committed to the repo before the outage, not authored during it.

A producer pushes 5,000 events per second into a consumer that manages 3,000. Someone puts an unbounded queue between them to "smooth the burst." Nothing is smoothed: a queue without a bound is an outage deferred. The queue grows by 2,000 items per second, latency grows with it, producers time out waiting for acknowledgments, their retries add more load, and eventually memory runs out and the process dies — losing everything the queue had accumulated. With Little's Law, queue_depth = arrival_rate × latency, an arrival rate above service rate means the depth goes to infinity; there is no steady state to wait for.

Backpressure is the discipline of telling the producer, continuously and through the transport itself, to slow down. It is older than microservices: TCP's receive window is backpressure — the receiver advertises credit, and when its window fills, the sender stalls at zero cost. The same idea recurs wherever you have a fast producer and a slow consumer:

The engineering rule attached to every row: the queue must be bounded and its fullness must be visible to whoever produces into it. In Go this is one select:

select {
case jobs <- j:
    // accepted
default:
    // bounded queue is full: tell the producer this is throttling, not success
    return ErrOverloaded
}

From that rejection point the signal travels upstream: your service returns 503 with Retry-After; the caller's client SDK treats it as a throttle and reduces its send rate; its own producers see their queues fill and slow further back. In Kafka the same loop runs through lag: consumers stop fetching when they fall behind (max.poll.interval.ms enforcement), lag becomes an autoscaling signal, and capacity chases demand — though you must remember autoscaling lags by minutes, so the in-process limits are what hold the line while pods spin up.

For in-process concurrency, the strongest pattern is adaptive: measure latency rather than error rate, and treat your own response time as the saturation signal. An AIMD controller (additive increase, multiplicative decrease — the same shape as TCP congestion control) ramps concurrency slowly upward and cuts it hard, say to 40 %, the moment p99 latency rises above your target. Netflix's concurrency-limits library implements exactly this, and it outperforms any fixed limit because the right number changes with dependency health, request mix, and time of day.

Note the boundary between the two remaining mechanisms, because interviews distinguish them explicitly. Backpressure says "slow down" to a producer that can hear you; load shedding says "no" to work that arrives anyway. Backpressure is first and preferable — it preserves the work — but it cannot reach a dumb client, a cron job from a partner, or traffic that has already crossed your edge. Those arrivals are handled by admission control and shedding, which is the next chapter's subject.

When you review a pipeline in a design interview, ask two questions about every queue and you will find the real risk almost immediately: what is the bound? and what does the producer see when the bound is reached? An answer of "we monitor queue depth" is not a bound — monitoring tells you about the flood after it arrives.

Traffic hits 3× your provisioned capacity. CPU sits at 100 %, every request now takes 8 seconds, and everything is failing — including the health checks that would have gotten you more replicas. This is the moment to abandon work deliberately, because the alternative is that the system abandons everything simultaneously. Load shedding is the admission-control answer: reject the least valuable work early so the most valuable work stays inside its SLO.

The word doing the work is early. Shedding happens at admission — before a thread is taken, before a query runs, before a queue slot is consumed. Once work is queued, the damage is done: under saturation, queueing latency dominates everything (an M/M/1 queue's wait grows as 1/(μ − λ), exploding as arrivals approach capacity). A system that accepts 100 % of requests and serves all of them at 8 seconds has failed harder than one that accepts 70 % and serves those at 300 ms, because the 70 % includes your paying customers and the 100 % includes nobody's patience.

So you need classes, decided in advance:

Two mechanics implement it. The first is a load-shedding admission controller: a cheap check (current queue depth, in-flight count, CPU, or rolling p99) runs before any expensive work, and when the threshold is crossed the controller answers 503 immediately for everything below the priority line. The check must be cheap and non-blocking — an admission controller that waits for a lock during saturation is a bottleneck at exactly the wrong time. Google's frontend design describes the principle plainly: shed load before the queue builds, because after that it is latency, not load, you are managing. The second mechanic is deadline awareness: if now + expected service time > request deadline, do not accept the request — you would spend real capacity producing a response nobody will read. Propagating deadlines from the originating caller down the chain makes that check possible, and it also lets deep layers skip work whose deadline has already expired (wasted-work elimination), which is free capacity during an incident.

Priority must survive the queue, or you get a priority inversion where background exports starve checkout. Use per-class bounded queues with the critical class served first, and reserve a slice of every pool for critical traffic — the bulkhead partitions from Bulkheads and Rate Limiting become priority-partitioned under this scheme. Watch the interaction with retries: a shed 503 that triggers aggressive client retries converts your careful shedding back into a storm, so the response must carry Retry-After and the retry budget from the first topic must be small enough that shed traffic cannot amplify.

Response semantics matter for the same reason they mattered for 429: shed work gets 503 (or 429 for per-tenant limits), not a 200 carrying an error page — clients and load balancers only route around you if you tell the truth in the status line, and a health-checking balancer must never see an overloaded node claim health.

Instrumentation is not optional: export shed counts per class as a first-class metric and alert on them. Shed rate is the most honest overload signal you can have — it means the system knew its limits and protected the core. Teams that never see shed counters in a dashboard are usually discovering their limits the other way, during an outage they did not control.

You can reason about these thresholds forever and still not know them. The way you find out what your degradation ladder, your shed thresholds, and your breaker settings actually do under pressure is to apply the pressure on purpose — which is exactly what Chaos Engineering and Game Days formalizes.