Parallel Work with errgroup

A service enriches each incoming request by fetching five upstream APIs in parallel. The rules: fail the whole request on the first upstream failure, and stop the remaining fetches as soon as failure is certain — waiting for the other four after one has 503'd wastes seconds the client will not forgive.

The hand-rolled version needs five moving parts, and each is a place to be subtly wrong:

var (
	wg   sync.WaitGroup
	mu   sync.Mutex
	first error
)
ctx, cancel := context.WithCancel(parent)
defer cancel()

for _, u := range urls {
	wg.Add(1)
	go func(u string) {
		defer wg.Done()
		if err := fetch(ctx, u); err != nil {
			mu.Lock()
			if first == nil {
				first = err
				cancel() // tell the siblings to stop
			}
			mu.Unlock()
		}
	}(u)
}
wg.Wait()
if first != nil {
	return first
}

Now audit what could have gone wrong instead: storing first without the mutex (data race, caught only under -race if at all); calling cancel() unconditionally so the last error wins instead of the first; forgetting cancel() entirely so the derived context outlives the request and leaks its timer; incrementing the WaitGroup after the goroutine starts (rare early return → Wait fires early). Four independent correctness hazards, every one of them invisible in review unless you know to look.

golang.org/x/sync/errgroup collapses the whole arrangement into its standard shape:

import "golang.org/x/sync/errgroup"

func fetchAll(urls []string) error {
	g, ctx := errgroup.WithContext(context.Background())

	for _, u := range urls {
		g.Go(func() error {
			return fetch(ctx, u) // siblings stop when any fetch fails
		})
	}
	return g.Wait() // first error, or nil if all succeeded
}

The signature states the contract: every g.Go function returns an error; g.Wait returns the first non-nil one; the derived ctx is cancelled the moment any function fails. The mutex, the error bookkeeping, and the cancel-call discipline are gone because the group owns them. What errgroup does not hide is ordering: WithContext must be called before the g.Go calls, the ctx passed into the closures must be the derived one (using the parent defeats sibling cancellation), and g.Wait must be called exactly once — those three rules cover nearly every errgroup bug in the wild.

errgroup's value is its precise semantics, and interviews probe them directly. Three sentences define the whole contract:

1. The first non-nil error cancels the derived context. Siblings that check ctx (via http.NewRequestWithContext, select on ctx.Done(), or a context-aware driver) stop promptly; siblings that ignore the context keep running until they finish naturally — cancellation is cooperative, exactly as in Cancellation with context.
2. g.Wait returns that first error. Errors returned after the first are recorded nowhere — first-failure-wins is a deliberate policy, not an oversight. If you need every failure, collect them yourself (the errors.Join pattern from Error Handling) or carry errors on a results channel instead.
3. The derived context is also cancelled when Wait returns — even on success. This is the gotcha that bites in production: after g.Wait, the ctx you handed to children is dead.

Rule 3 has teeth with HTTP:

bodies := make([]string, len(urls)) // NOT []*http.Response

g, ctx := errgroup.WithContext(context.Background())
for i, u := range urls {
	i, u := i, u // loop-var capture explicit for pre-1.22 Go
	g.Go(func() error {
		req, _ := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
		resp, err := http.DefaultClient.Do(req)
		if err != nil {
			return fmt.Errorf("fetch %s: %w", u, err)
		}
		defer resp.Body.Close()
		b, err := io.ReadAll(resp.Body) // read INSIDE the goroutine
		if err != nil {
			return fmt.Errorf("read %s: %w", u, err)
		}
		bodies[i] = string(b)
		return nil
	})
}
if err := g.Wait(); err != nil {
	return err
}

Two structures worth naming. Results go into an indexed slice, not an append. Each goroutine writes only bodies[i] — its own slot — and distinct elements of a slice are distinct memory, so no mutex is needed; a shared append would race exactly like the hand-rolled code's first field. Read the body inside the goroutine. Returning *http.Response looks useful but the body read happens after Wait cancelled the request context, and reads on a cancelled request fail or truncate. Compute the durable value (bytes, parsed struct) before returning; keep lifetimes inside the group.

The per-goroutine discipline this reduces to: take ctx from the group, do context-aware work, write only your own index, return a wrapped error with %w, and never touch a sibling's slot. Five rules, no locks.

Launching 5,000 goroutines through g.Go recreates the unbounded-spawn problem from Worker Pools — errgroup changes failure semantics, not physics. SetLimit turns the group into a bounded pool:

g, ctx := errgroup.WithContext(ctx)
g.SetLimit(16) // at most 16 goroutines live at once

for _, id := range itemIDs {
	id := id
	g.Go(func() error {
		return process(ctx, id) // g.Go BLOCKS here while 16 are running
	})
}
if err := g.Wait(); err != nil {
	return err
}

With a limit set, g.Go blocks until a slot frees — the calling goroutine becomes the dispatcher, and memory stays proportional to the limit rather than the task count. TryGo is the non-blocking variant: it returns false instead of waiting, for callers that must react to saturation (requeueing, shedding) rather than stall. Set the limit before the first g.Go, and size it by workload class exactly as a pool is sized: GOMAXPROCS for CPU work, the dependency's concurrency budget for I/O.

Knowing when not to use errgroup is the advanced half of the topic. The honest decision table:

The distinction in the first two rows is the one interviewers listen for: errgroup models a batch of work with an outcome — N calls, one result, abort-on-failure — while a pipeline models a stream with ongoing throughput, where one bad item should be reported without necessarily stopping every item after it. The cancellation plumbing errgroup gives away for free is precisely the layer the hand-rolled fan-out pattern from the previous topic lacked; taking it does not change the topology, only who is responsible for the error and when the siblings hear about it.