Circuit Breaker Pattern

Your orders service calls the pricing service. Pricing is healthy at 40 ms, then a bad deploy pushes p99 to 8 seconds. Timeouts from Timeouts and Retries stop the calls from hanging forever, but they do not stop them from occupying a worker for the full 2 seconds each — and your retry policy multiplies that by three. Every slot in the orders thread pool is now spent waiting for pricing, so orders stops answering even its own local endpoints. The dependency is sick; your service is the one that dies.

A circuit breaker changes the question from "how long do I wait?" to "should I try at all?" It is a small state machine wrapped around each remote call, and it has exactly three states.

While closed, the breaker counts failures — thrown exceptions, timeouts, selected 5xx responses — inside a sliding window. When the failure rate crosses the threshold, it opens. From that instant, calls do not reach the network at all: they throw CircuitBreakerOpenException (Resilience4j), return a fallback (Polly), or get 503 from the gateway. The win is double. Your worker slots are released instantly instead of burning 2 seconds apiece, and the struggling dependency stops receiving traffic it cannot handle, which is often the only thing standing between it and recovery.

After a cooldown — the wait duration in open state, commonly 5–30 seconds — the breaker moves to half-open. Here is the detail candidates most often miss: half-open does not mean "let everything through and see." It admits a fixed allowance, for example 5 permitted calls or a trickle of a few per second. If those probes succeed within the allowance, the breaker closes and normal traffic resumes. If a single probe fails, it snaps back to open, usually with a fresh (sometimes longer) cooldown. Probes are deliberately few because the dependency has not proven anything yet; a recovering service slammed with your full request rate immediately after the timer expires is a service you are actively keeping sick.

A concrete configuration from Resilience4j makes every knob tangible:

CircuitBreakerConfig cfg = CircuitBreakerConfig.custom()
    .failureRateThreshold(50)                       // % failures that trips a closed breaker
    .slowCallRateThreshold(50)                      // % slow calls that trips it too
    .slowCallDurationThreshold(Duration.ofMillis(500))
    .slidingWindowType(SlidingWindowType.COUNT_BASED)
    .slidingWindowSize(20)                          // evaluate the last 20 calls
    .minimumNumberOfCalls(20)                       // ignore noise below this sample size
    .waitDurationInOpenState(Duration.ofSeconds(10))
    .permittedNumberOfCallsInHalfOpenState(5)       // probe allowance
    .automaticTransitionFromOpenToHalfOpenEnabled(true)
    .build();

Note what the state machine does not do: it never blocks a thread, and open-state rejection is as cheap as an if. That matters because a breaker that costs anything meaningful under load would be unusable exactly when you need it.

In interviews you will be asked to draw the state machine and explain the transitions cold. Draw closed → open on the threshold, open → half-open on the timer, half-open → closed on successful probes, half-open → open on a failed probe. Then say the sentence that separates seniors from the rest: the breaker is a feedback loop between your traffic and someone else's health — it trades your immediate availability (a fast error now) for the chance that both systems recover instead of neither.

The breaker logic above is easy to code and easy to misconfigure. Get the thresholds wrong and you get one of two failure modes: a breaker that trips on noise (your own users see 503 because two requests failed out of three) or a breaker that never trips at all because the window is too large to notice a real collapse. Configuration is where the pattern earns its keep.

Start with what feeds the counter:

Two lines of that table carry most of the weight. Timeouts must count as failures: a pricing service that always answers in 8 seconds with 200 OK will never trip a pure error-rate breaker, yet it is destroying your latency SLO. That is why slowCallRateThreshold plus slowCallDurationThreshold exists — a dependency that never fails but never returns in time is still a failure, from your caller's point of view. Conversely, a 404 from a perfectly healthy service is not evidence the service is broken; counting client errors trips breakers for reasons the dependency cannot fix.

Next, the sample-size guard. A failure rate of 50 % means nothing when the denominator is small: on a low-traffic dependency, two bad requests in a row is 100 % and would open the breaker permanently. minimumNumberOfCalls (Resilience4j) or an equivalent minimum-request threshold (Netflix Hystrix used 20) says: evaluate the rate only after enough evidence exists. A breaker with a count-based window of 20 and a minimum of 20 calls reacts to the last 20 requests — fast and proportional. A time-based window of 30 seconds gives you predictable semantics under varying traffic: the rate is always "the last 30 seconds," regardless of volume. Count-based is simpler and self-tuning under load; time-based is easier to explain to a stakeholder. Pick deliberately and write it in the runbook.

Dwell time is the anti-flap control. Without it, a breaker that closes the instant the dependency is healthy again hands the recovering service 100 % of your traffic, watch it stumble, re-opens, and oscillates — a sawtooth that is worse than either state. waitDurationInOpenState plus a small half-open allowance converts the recovery into a ramp. Some teams grow the cooldown on repeated trips (open → 10 s, again → 30 s, again → 2 m) so a dependency that keeps failing gets longer and longer rest.

Probe volume deserves the same care: too few probes and recovery detection is slow (your service stays degraded long after the dependency healed); too many and you re-create the thundering herd the breaker existed to prevent. One to five calls per half-open period, or a small percentage of traffic, is the usual range.

Finally, scope the breaker correctly: per dependency and, in a fleet, per upstream instance. One dead replica behind a load balancer should trip the breaker on that host only — a circuit breaker keyed on the service name lets one bad pod open the breaker for all five good ones. The infrastructure cousin of this idea is load-balancer outlier detection, which ejects the unhealthy host for the same reason.

Interviewers like the question "how would you tune a breaker that keeps flapping?" The answer order matters: verify what counts as a failure (are timeouts included?), raise the minimum sample size, extend the dwell time, reduce half-open probe count — in that order, each addressing a different cause.

A breaker that trips correctly but has no answer for "so what does the caller get?" has only moved the error around. Three decisions remain: where the breaker lives, what the open state returns, and how it composes with your retry policy.

Placement is a granularity-versus-control trade-off. In-process, in the client library (Resilience4j, Polly, Go's gobreaker): per-host granularity, sub-millisecond rejection, language-specific tuning — but you deploy the policy with the code and every team configures its own. At the API gateway or service mesh (Envoy outlier detection, AWS ALB): one place to configure, language-agnostic, works for every service behind it — but it only sees one hop, so it cannot protect internal fan-out chains, and its notion of failure is usually HTTP status plus consecutive ejections, not your application's notion of failure. Sidecars sit between the two: mesh-wide policy, per-cluster scope. Teams in interviews usually pick in-process for internal calls and gateway-level for public edges, and can justify both.

The open state needs a designed answer, not an accidental one. Options, in order of preference when available: serve stale-but-labeled cached data; serve a default or reduced response; return an explicit 503 with Retry-After so a well-behaved client comes back later; enqueue for asynchronous completion when the operation tolerates latency. The rule that matters: the fallback must be cheaper than the call it replaces. A fallback that calls a different slow dependency, hits an unbounded queue, or acquires the same contended pool recreates the original failure with extra steps. Test the fallback path as hard as the happy path — it is the code that only runs during incidents.

This is also where you draw the line between complementary patterns:

• Retry decides how many times to try within one caller's budget (Timeouts and Retries).
• Breaker decides whether trying at all is rational right now.
• Bulkhead decides how much of your own capacity a struggling dependency may occupy (Bulkheads and Rate Limiting).
• Rate limit / load shed decides whose request gets served when everything is contended.

Decide explicitly how retries interact with breaker state: either the retry loop is inside the breaker's scope (each attempt counts separately, so an aggressively retrying caller trips its own breaker — useful) or outside (the breaker sees one logical call per request — simpler to reason about, but a 3× retry multiplier hides inside each sample). Whichever you choose, configure attempts down to 1–2 while half-open; a probe allowance of 5 combined with 3 retries per probe is 15 requests hitting a dependency you have not yet cleared.

Give the open state a latency budget as strict as the call it replaces. If the fallback reads a cache, that read needs its own short timeout; if it renders a default page, the render must not touch the same contended pool. A breaker whose fallback takes 300 ms has not failed fast — it has converted a 2-second dependency timeout into a 300-millisecond failure that shows up in your latency chart instead of theirs.

Operational blind spot: a breaker with no telemetry is invisible until it hurts. Export state transitions, rejected-call counts, fallback counts, and probe outcomes as metrics, and alert on breaker state rather than only on error rates — an open breaker is an early warning that fires before your downstream error-rate dashboard moves, because it is consuming failures so your users do not.

Teams adopting breakers often ask how to prove they work without waiting for a real outage. The honest answer: you inject the failure deliberately. That is the discipline covered next in Chaos Engineering and Game Days, where a hypothesis like "the pricing breaker opens within 10 s and checkout keeps serving from cache" becomes a test you can actually run.