Fan-Out and Fan-In

A data pipeline reads records, applies an expensive enrichment call per record, and writes results. Built as a straight three-stage chain from pipeline fundamentals, it looks clean and runs at exactly one enrichment at a time — because fan-out is the missing piece: one fast stage feeding one slow stage makes the slow stage the entire pipeline's throughput, no matter how fast everything else is.

Fan-out is the fix: give the slow stage multiple workers reading from the same input channel.

func fanOut(in <-chan Record, n int) []<-chan Enriched {
	outs := make([]<-chan Enriched, n)
	for i := 0; i < n; i++ {
		out := make(chan Enriched)
		outs[i] = out
		go func() {
			defer close(out)
			for r := range in { // ALL workers range over the SAME channel
				out <- enrich(r)
			}
		}()
	}
	return outs
}

The design rests on one channel fact: a receive on a shared channel delivers each value to exactly one receiver. Eight workers ranging over one input do not see eight copies of each record — they compete for them, and the runtime hands each record to whichever worker is parked and waiting. That is load balancing for free, with no work-stealing algorithm and no queue-per-worker. Records distribute roughly evenly because each worker takes the next one the instant it becomes available.

Termination is inherited rather than engineered. The upstream producer closes the input when it is done, and because every worker is ranging over that same input, all N workers end their loops on that single close, each closing its own private output via defer. No worker counts jobs; no coordinator is told how many records existed. The whole fan-out layer's lifetime is stitched to the input's close — which is why the ownership rules from Closing Channels are prerequisites, not trivia: close the shared input once, from its producer, after the last send.

What makes fan-out advanced is not the launch loop — it is reasoning about its conditions. Fan-out pays off when jobs are independent and expensive relative to channel overhead. It hurts when jobs are trivial (goroutines spend their time handshaking instead of computing), when jobs share mutable state (now you need the mutex you were trying to avoid), or when the real bottleneck is downstream: replicating the enrichment stage eight ways while the writer stage still accepts one record at a time just moves the choke point. Parallelize the stage that is actually slow — worker pool sizing applies per stage.

Eight workers now produce eight channels of results, and the next stage — aggregation — accepts one. Fan-in merges them, and the merging goroutine has exactly two duties: forward values from every input, and close the merged output after the last input closes, exactly once.

func fanIn(channels ...<-chan Enriched) <-chan Enriched {
	var wg sync.WaitGroup
	merged := make(chan Enriched)

	wg.Add(len(channels))
	for _, c := range channels {
		go func(c <-chan Enriched) {
			defer wg.Done()
			for v := range c { // ends when this input closes
				merged <- v
			}
		}(c)
	}

	go func() {
		wg.Wait() // every input channel drained
		close(merged)
	}()

	return merged
}

Each input gets a forwarding goroutine; each forwarder's range ends when its input closes; the closer goroutine counts them all with a WaitGroup and closes merged once, after the final forwarder exits. The consumer writes for e := range fanIn(out1, ..., out8) and cannot tell the stream was ever parallel. Every piece is a rule already established — single closer, coordinator after WaitGroup, range for termination — but their composition here is worth reading slowly, because it is the template for merging arbitrary channel counts.

Notice the closure-capture detail that production reviews catch: c is passed as a goroutine argument rather than read from the loop variable. Go 1.22+ fixed the classic loop-variable capture bug, but passing the channel explicitly remains correct on every version and states intent — this goroutine forwards that channel.

Ordering is where fan-in choices fork. The merge above is unordered: whichever input yields a value first is forwarded first, so output order depends on worker timing. For most aggregation (sums, counts, dedup) that is fine and fastest. When order matters — replaying events by sequence — two approaches exist:

• Reorder after merging: each item carries its sequence number; the merger emits into a min-heap and forwards only when the next expected sequence is available. Flexible, costs a heap and holds items back until stragglers arrive.
• Slot buffer: launch N workers for item i into results[i % N]-style fixed slots, wait for slot i before emitting it. Deterministic and simple when items are pre-indexed; a slow worker stalls its slot — and effectively the whole stream — which is the price of order.

Unordered is the default for throughput; ordered is a requirement you accept latency for. Choosing either without knowing what the consumer needs is the actual mistake.

The pipeline now has fan-out and fan-in, so the natural question is why the p99 did not move. Four structural causes cover most real cases.

The stage was not the bottleneck. Fan-out multiplies that stage's capacity. If the writer downstream still serializes — one INSERT per record on a single connection — enrichment eight ways just feeds the writer a queue eight times faster, and latency moves to the writer's input buffer. The diagnostic is per-stage timing: instrument each stage's input queue depth (len(chan) sampled on a timer) and put the workers where the depth is pinned. This is the pipeline version of the buffered-channel lesson: a deep queue marks the constraint, everything upstream of it is just waiting.

The fan-out was wider than the work. With cheap jobs, channel handoff costs (park, wake, copy) rival the job itself, so N workers amplify overhead. Rule of thumb: if enrich finishes faster than roughly ten times a channel round-trip, the parallel version loses to a single worker — measure rather than assume.

The shared input became a coordinator cost. Many workers all pulling from one channel occasionally contend on the same queue structure under extreme load. At modest N (tens) it is noise; at N in the hundreds, splitting input into per-worker channels with a dispatcher — essentially several pools fed by a splitter — relieves it. That is architecture-level tuning; reach for it only after profiling shows the contention.

Errors and cancellation are not wired yet. This is the structural gap the pattern itself cannot close. One worker's enrich failure currently has nowhere to go — it either panics the goroutine (crashing the process) or gets swallowed, while the other seven keep working through a run whose results the consumer will discard anyway. Two established answers, each with a shape:

// (a) results carry errors — consumer decides, like the worker pool pattern
type Enriched struct {
	Rec Record
	Err error
}

// (b) group semantics — first failure cancels the rest (next topic)
g, ctx := errgroup.WithContext(parent)
// each worker: g.Go(func() error { ... })

Option (a) preserves the stream when partial success is meaningful — some records failing enrichment should not void the others. Option (b) fits when any failure invalidates the batch and the remaining workers should stop wasting work — the job of errgroup, which packages exactly the hand-rolled wait-cancel-collect code this pattern would otherwise grow into. And cancellation itself — draining stops, workers exit, no forwarder parked on a full merged channel — needs the signal plumbing from context. Fan-out/fan-in supplies the data topology; error and shutdown semantics are the layers this phase adds next, one topic at a time.

It helps to see all the moving parts assembled once, with the ownership arrows made explicit, before committing the pattern to memory:

func pipeline(records <-chan Record, workers int) <-chan Summary {
	// stage 1: fan out the expensive call — each worker owns its output close
	enriched := make([]chan Enriched, workers)
	for i := 0; i < workers; i++ {
		enriched[i] = make(chan Enriched)
		go func(ch chan Enriched) {
			defer close(ch)
			for r := range records { // shared input: one record goes to one worker
				ch <- enrich(r)
			}
		}(enriched[i])
	}

	// stage 2: fan in — one forwarder per worker, one closer for the stream
	merged := make(chan Enriched)
	var mergeWG sync.WaitGroup
	mergeWG.Add(len(enriched))
	for _, ch := range enriched {
		go func(ch <-chan Enriched) {
			defer mergeWG.Done()
			for e := range ch { // ends when this worker's output closes
				merged <- e
			}
		}(ch)
	}
	go func() {
		mergeWG.Wait() // every forwarder has drained its input
		close(merged)  // exactly once, after all worker outputs are done
	}()

	// stage 3: consume the single stream
	out := make(chan Summary)
	go func() {
		defer close(out)
		for e := range merged {
			out <- summarize(e)
		}
	}()
	return out
}

Read the closes as a chain, because that chain is the correctness argument: the producer of records closes it → all N enrichment workers' ranges end → each closes its private output → the merge forwarders' ranges end → the merge closer closes merged → the summarizer's range ends → it closes out → the final consumer's range ends. Every goroutine has a path to exit, and nobody waits on a count that someone forgot to increment. One broken link — an unclosed records, a worker that returns early without closing its output — parks an entire downstream stage forever.

What you should be able to say about this pattern in an interview is not the code but the judgment: fan-out trades latency for N× resource use and a load-balanced shared queue; fan-in trades simplicity for either unordered speed or ordered buffering; the pair only helps when the parallelized stage is measured to be the constraint; and the data-topology pattern stops short of error and cancellation semantics, which need errgroup and context layered on top. When those layers land, the shape barely changes — the workers' bodies gain a context check and an error return — which is the mark of a pattern built on the right primitives to begin with.