Distributed Monolith Pitfalls

The retrospective opens with a slide: "Order API v2.14 deploys require Billing 3.9, Customers 2.6, and Inventory 4.1 to be upgraded first; the coordinated release window is Thursday 02:00 UTC." Six processes, three Kubernetes clusters, one deploy train. Everyone waits for the same window, one failure rolls back all five, and the on-call channel has a name for the window: "release roulette." This is a distributed monolith — separate processes that still change, release, and fail as one unit — and it is the worst possible outcome: you pay for distribution and keep all the coordination.

The definition has teeth: a distributed monolith requires coordinated releases despite running as separate processes. Coordination takes four forms, and you can audit for each:

Notice which forms the org chart hides. Each team believes it owns its service end to end; the release calendar tells another story. The cheapest audit is data you already have: pull the last quarter of deployments and measure release correlation — of the times service A deployed within an hour of service B, what fraction were the same change (linked PRs, same release ticket)? Below 10%: independent ships. Above 40%: you built a distributed monolith with extra YAML. The second metric is simpler: how many services appear in a typical rollback? One is the point of the architecture; five means a failure's blast radius was never actually isolated. A third check costs nothing: ask two teams who owns the Thursday window. If the answer is "we all do," the calendar already knows what the metrics will confirm.

Why this matters beyond embarrassment: the resilience promise is inverted. In a monolith, a slow dependency is a stack frame you can degrade locally; in a properly split system, one service's failure stays in its blast radius. In a distributed monolith, a slow member still pins the synchronous chain (as in Inter-service Communication), and its fix requires the Thursday window — so the incident is both wider and slower to repair. Teams in this state often respond by adding more services, which deepens the train. The correct reading is that the boundaries were never real: the split drew deployment boxes around code that still shares one model, one library, and one release reality.

Interview framing: this is a favorite advanced question — "what is a distributed monolith and how would you know if you had built one?" Answer with the definition, two of the four forms, and a measurement (release correlation, rollback blast radius). Then signal the next move: the two most common causes — shared libraries and synchronous temporal coupling — each have concrete escapes, which is where the next two chapters go.

common-model is 40,000 lines and every service depends on it. It holds Customer, OrderStatus, the money type, the event base classes, and — inevitably — CustomerView, which gained a loyaltyTier field eighteen months ago that exactly two services use. Adding a field means bumping the major version, which means every service must adopt the new version in the same release cycle or compile against a version with a different Customer definition. The library, built to reduce duplication, has become the mechanism that forces joint upgrades: shared libraries versioned in lockstep across services force joint releases, and the jar is where your autonomy went to die.

The deep problem is not the tooling, it is that a shared library distributes code where the system needed contracts. Two services linking the same class file still evolved at different paces, so the class became a negotiation table. Escapes, ranked by how much model they let you share:

The first escape is the default answer for events and APIs: the wire contract is the boundary, and code generation means each service's OrderPlaced type is its own, compiled against its version of the schema. Backward-compatible evolution (add optional fields, never change tags) means old and new coexist — which is precisely what a shared jar cannot do, because one class definition cannot be two versions at once in one JVM dependency graph.

The second escape — duplication — makes purists flinch and is often right. Thirty lines of money arithmetic duplicated across four services costs 120 lines and four places to fix a rounding bug. The alternative (the shared library) costs a coordinated release every time anything in it changes. Duplication is a local cost; lockstep is a systemic one. The discipline is making the duplication explicit: a header comment with origin and a link to the owning module's source, so future readers know it was a decision, not an accident. When the copy changes three times, that is your evidence-based trigger to promote it to a versioned package — not before.

How to dig out of an existing mess: inventory who imports what in common-model, split it by change frequency (the money type is stable; the CustomerView DTO is not), publish the volatile parts as schemas, freeze the stable parts, and put a deprecation clock on the rest. Measure progress the same way as the previous chapter: count joint upgrades per quarter — it should be falling. A library is allowed to be shared; what it is not allowed to do is make service A's deploy contingent on service B's calendar, because that is the lockstep failure wearing a pom.xml.

Diagnosis without cure is just a better incident report. Escaping the distributed monolith means attacking the two couplings that survive reorganizations — temporal and release — and then proving with data that independence is real.

Temporal coupling first, because it is what users feel. A caller that needs loyalty-service up right now for every request has synchronized their fates whether or not anyone chose to. The redesign follows the answer question:

Does the caller need the answer in this request?
  yes, and the member is critical        → sync with tight deadline + honest failure;
                                           accept the joint fate explicitly, in writing
  yes, but degraded is fine              → sync behind a circuit breaker with a
                                           pre-agreed degraded answer
  no                                     → replace the call with a published event;
                                           caller ships without callee's deploy

Each "no" converts a temporal edge into a contract edge: the producer publishes OrderPlaced and deploys whenever it likes; the consumer catches up on its own schedule. The price — eventual consistency and a delivery contract — was analyzed in the communication topic; the gain is that release ordering constraints disappear from that edge entirely.

Release coupling second. Joint releases persist after async conversion when contracts are negotiated by humans ("we changed the payload, so coordinate the deploy"). Kill that with mechanical compatibility: additive-only schemas, a registry, and consumer-driven contract tests in CI — the producer runs its consumers' expectations on every PR, so "will their code break?" is answered by a red build, not by a Thursday negotiation. When contract tests pass, deploys need no ordering at all, and the release window becomes an ordinary time of day.

Then measure, because autonomy claimed is not autonomy delivered:

Any one metric can mislead (a quiet quarter looks independent), so read them together — and be honest about the metric that may not move: if the underlying work is genuinely co-changing, no amount of tooling will make the teams independent, because they built one cohesive unit. At that point the mature answer is merging services back — the same data-driven reversal from When Microservices Make Sense. Admitting that a split should be undone is a stronger architectural position than defending boxes on a diagram.

The interview-ready summary: distributed monolith = coordinated releases across separate processes, detected by release correlation and rollback blast radius; the two usual causes are lockstep shared libraries (fix: share contracts, not code) and synchronous temporal coupling (fix: event edges where no answer is needed, contract tests where there is); and the proof is a dashboard showing lead time falling while deploy correlation falls with it. That answer demonstrates you have run the loop — measure, cut, re-measure — instead of merely naming the anti-pattern.