Event Sourcing

The bank's ledger stores balance = 4300. Ask it why the balance is 4300 and it cannot answer. An event-sourced ledger stores the answer instead of the summary: Deposited 5000, Withdrew 400, Refunded 100 — an append-only, totally ordered log per entity from which the current state is derived by folding the events through deterministic handlers. The row is a cache of the story; the story is the truth.

events for account-771 (append-only, ordered):
  [0] AccountOpened{owner:'a'}            → state: open, balance 0
  [1] MoneyDeposited{amount:5000}         → balance 5000
  [2] MoneyWithdrawn{amount:400}          → balance 4600
  [3] RefundReceived{amount:100}          → balance 4700

load: replay [0..3] through handlers → in-memory aggregate → respond to a command
mutate: validate command against current state → append new event → done

The write path deserves precision because it differs from CRUD in one critical way: commands are validated against the current state, and the result is a new immutable event, not an UPDATE. Two clients withdrawing concurrently both read balance 4700, both validate, and both append — a lost update. Event sourcing fixes it with optimistic concurrency on the stream version: the append carries expectedVersion = 3, and the store rejects the second writer (WrongExpectedVersionException), who re-reads, re-validates, and retries. That check is the aggregate's serialization point — event sourcing does not remove concurrency control, it expresses it as stream versioning.

What you gain by paying the storage and complexity price, stated concretely:

• A complete audit trail for free — regulators, disputes, and incident forensics get the exact sequence with timestamps and causation, rather than a last-value snapshot and hope.
• Temporal queries: 'what did the customer's cart look like at 14:32?' is a replay to a position, not a reconstruction from backups.
• Debugging by replay: reproduce production state on a dev machine by replaying a stream through a debugger — the failing state is a function of inputs, and you have all of them.
• Decoupled projections: any read model (CQRS views, search indexes, downstream services) is a fold of the same log, and a new one is a backfill — the two patterns pair naturally (CQRS Pattern).

The costs to state just as plainly: the write model must be expressible as facts (an inventory system whose truth is a relationship — foreign keys, cross-entity constraints — fits poorly); eventual projection lag becomes user-visible; and the log's schema is forever, which is the versioning problem chapter two handles. Also note the failure mode that catches every beginner: an event is a fact about the past, never a command — MoneyWithdrawn not WithdrawMoney — and handlers must be deterministic (no now(), no random, no external calls inside replay), with time and side effects captured in the event data at append time. A non-deterministic handler makes replay — the pattern's entire value proposition — untrustworthy. A quick sizing exercise also keeps expectations honest: 90,000 ten-event streams and 900,000 one-event streams occupy the same rows; it is stream length, not entity count, that drives replay cost, and that number decides whether the next chapter's snapshots are an optimization or a necessity.

The event MoneyDeposited{amount: 4000} was written 18 months ago, in 40 million streams. Today's code needs MoneyDeposited{amountMinor: 400000, currency: 'USD'} — a rename and a unit change. In a CRUD database this is a migration. In an event-sourced system the old events are the system of record: they cannot be edited, and every stream ever written must still replay into today's code. The problem is therefore not migration but evolution of a permanent history, and the standard solution has two halves: versioned event types and upcasting.

raw stored event:      MoneyDeposited.v1 { amount: 4000 }
                              │  loader sees type + version in the envelope
                              ▼
upcaster chain:        v1 → v2   { amountMinor: 400000, currency: 'USD' }
                              │  (pure function, in the application, runs at read time)
                              ▼
on-disk format:        UNCHANGED forever
in-memory format:      today's v2 — handlers only ever see current versions

The mechanics, precisely. Each stored event carries a type name and version (MoneyDeposited.v1, or Protobuf-style field evolution). At load time, the repository passes the raw payload through upcasters — pure functions registered per version step (v1→v2, v2→v3), applied in chain — so handlers are written once against the newest shape and never know the history exists. Old data is upgraded in memory at read time, never rewritten on disk: writing terabytes of history for a field rename is a non-starter, and rewriting would destroy the audit property that justified the system.

Design rules that separate durable systems from demos:

• Additive evolution where possible. Adding an optional field is as safe here as in messaging contracts — new events carry it, old events lack it, upcasters supply the default. Breaking changes (semantic, not just structural) get a new event type, not a reinterpreted old one (MoneyDeposited.v2 coexists; old streams simply never emit it).
• Semantic upcasts are the danger. amount → amountMinor is mechanical. But 'we changed what Deposited means (now excludes refunds)' is a semantic break: the upcaster must reproduce the old meaning faithfully, and the only safe inputs are the facts stored in the old event. Write upcasts as unit tests over captured real payloads, not synthetic ones.
• Upcasters run in the application, not the database — they are version-control-managed code with tests, executed on every load of an old event. Their cost is on the read path, so keep them cheap; heavy transformations belong in projections that can be rebuilt.
• Snapshot versioning is the same problem at the aggregate level. A snapshot of a stream is schema-bound; either version snapshots and upcast them too, or discard-and-rebuild a stream's snapshot when its version falls behind — never block loading because a snapshot is old.

Also decide what the projection side does with the same pressure: projections fold current-version events into views, and a projection's schema can evolve by rebuild (drop, re-code, replay) rather than upcast — which is why the ecosystem pairs event sourcing with CQRS: the log stays frozen and versioned; the views stay disposable. Interview-ready summary: history is immutable, so evolution happens at read time (upcasters) and at the type level (new event versions), with semantic changes tested against real old payloads; anything that can be rebuilt is rebuilt instead of migrated. The final chapter covers live-stream operations: snapshots, streaming reads, and the limits of the model.

The payment stream now has 90,000 events. Loading the aggregate for every command replays all of them — 40 ms per command, and the checkout path notices. This is event sourcing's operational center of gravity: replay cost grows with stream length, so the design must bound it. The standard tool is the snapshot: a periodically persisted materialized state at a stream position, after which only the tail replays.

without snapshot:  replay events[0 .. 89999]              → 40 ms/command
with snapshot@88000: load snapshot + replay events[88001 .. 89999] → 0.5 ms/command

trigger: every N events (e.g. 200–1000) or on a time budget
failure policy: snapshot missing/corrupt ⇒ fall back to full replay (snapshots are
               a cache — losing them costs latency, never data)
space policy:   keep the latest K snapshots, GC older ones

Two disciplines make snapshots safe. First, snapshots are derived state: never load-only. Every reader path must tolerate their absence — a system that refuses to boot because a snapshot is unreadable has turned a cache into a dependency. Second, snapshot writes are asynchronous to the command path (write event, ack, snapshot in background) so a slow snapshot store cannot slow the write.

The second operational pillar is streaming reads for projections. Projections do not load aggregates; they tail the log — pulling batches by position, checkpointing their offsets, partitioned by stream so per-entity order holds (Message Queues and Brokers mechanics return again). Long-lived projections need idempotent application and version guards, exactly as CQRS demands; a projection lag alarm is the freshness SLO of the whole system.

The honest limits, which interviews reward you for stating:

• Cross-aggregate invariants have no home. Two streams cannot share a transaction; 'no double-sell' must be designed into a single aggregate's validation or handled with process managers/sagas and compensation. Model aggregates around the invariants first — that is the hard design work event sourcing does not remove.
• The query problem. 'All accounts with balance over X' requires a projection; there is no SELECT over history. Every question you have not anticipated needs a new fold and a backfill — powerful, but it is engineering time.
• Schema permanence and retention. Events live forever (or until archive), so PII erasure becomes a tricky exercise (archive + strip fields via upcasting, or tokenize at append time), and storage grows monotonically — capacity plan the log, not just the tables.
• Handler discipline is enforced by replay. Any non-determinism (time, randomness, external calls) poisons reproduction; side effects belong in reaction handlers keyed off events, outside the fold.

Architectural fit, delivered as the closing judgment: event sourcing earns its cost where the history itself is the value — financial ledgers, billing, compliance-heavy domains, collaborative editing, debugging-intensive systems — and drags everywhere else. Combined with CQRS it forms the full shape: an immutable, versioned log as the system of record; disposable, rebuildable read models; snapshots for speed; and a contract discipline on events that outlives every deploy. That contract discipline generalizes beyond event stores into public interfaces — REST Design and Versioning is where the roadmap applies versioning thinking to APIs.