Coordinating with sync.WaitGroup

The handler from the previous topic fired three lookups and slept 400 ms, hoping they would finish. On CI it returns empty strings; on a good day it wastes 100 ms per request; and nobody can explain what the number 400 has to do with the work. The replacement is a counter: sync.WaitGroup tracks how many goroutines are outstanding, and Wait blocks until the counter reaches zero.

func (s *Server) overview(r *http.Request) (Profile, Prefs, Quota, time.Duration) {
	start := time.Now()
	var (
		wg              sync.WaitGroup
		profile Profile
		prefs   Prefs
		quota   Quota
	)

	wg.Add(3) // register the three tasks BEFORE starting any of them
	go func() { defer wg.Done(); profile = s.fetchProfile(r.Context()) }()
	go func() { defer wg.Done(); prefs = s.fetchPrefs(r.Context()) }()
	go func() { defer wg.Done(); quota = s.fetchQuota(r.Context()) }()

	wg.Wait() // blocks until all three have called Done
	return profile, prefs, quota, time.Since(start)
}

The three calls to the API are simple, and each has one rule behind it:

• Add(n) before the goroutines start. The counter must already reflect the work before any completion can be reported.
• Done() in a defer inside the goroutine. Defer runs on every return path, so an early error return cannot strand the counter. Done() is just Add(-1); the zero value is usable immediately.
• Wait() in the goroutine that owns the count. It parks the caller until the counter hits zero. It does not cancel anything, collect results, or report errors — it only answers "are all of them done yet?"

Why the results are safe here is worth stating precisely: each goroutine writes a different variable, so no two write the same memory. Contrast that with the tempting var results []Result followed by results = append(results, r) inside goroutines — three appends to one slice header race with each other and can corrupt the slice entirely. The fix is to size the slice first and let goroutines write disjoint slots:

results := make([]Result, len(urls)) // allocate once
wg.Add(len(urls))
for i, u := range urls {
	go func(i int, u string) {
		defer wg.Done()
		results[i] = fetch(u) // each goroutine owns index i
	}(i, u)
}
wg.Wait()

Writing distinct elements of a slice concurrently is safe — the shared part is the backing array's structure, and nobody mutates the header. With Wait returning after the last Done, every write is visible to the reader, a guarantee the memory model attaches to WaitGroup itself: Done happens-before the corresponding Wait returns.

WaitGroups fail in exactly two directions, and both come from a miscounted counter.

Wait returns while goroutines still run. The cause is almost always Add inside the goroutine:

var wg sync.WaitGroup
for _, u := range urls {
	go func(u string) {
		wg.Add(1) // TOO LATE — main may reach Wait before this runs
		defer wg.Done()
		fetch(u)
	}(u)
}
wg.Wait() // may return while the counter is still 0

Timing decides the damage: if Wait executes before any goroutine starts, the counter is zero and it returns immediately — the caller then reads half-written results. The race is real but quiet, which is why it passes review. Rule: Add belongs in the spawning goroutine, before the go statement, never in the spawned one.

Wait never returns. The counter never reaches zero because some Done was skipped — a goroutine returned on an error path without defer, or Add(2) was paired with one Done, or Wait was called on a copy of a WaitGroup that nobody shares:

wg.Add(2)
go func() { defer wg.Done(); workA() }()
go func() {
	if err := workB(); err != nil {
		return // BUG: no Done — defer wg.Done() was missing
	}
	wg.Done()
}()
wg.Wait() // hangs forever if workB errors

The hang shows up as a stuck test with a goroutine dump pointing at Wait. Two habits eliminate the class: defer wg.Done() as the first line of every tracked goroutine, and never copying a WaitGroup — pass the pointer (go f(&wg) captures by closure anyway) since a copy is a different counter.

Reuse has one documented rule. After Wait returns you may start a second batch with fresh Add calls, but new Adds must not overlap a pending Wait on a zero counter:

If a WaitGroup is reused to wait for several independent sets of events, new Add calls must happen after all previous Wait calls have returned.

A WaitGroup answers one question — how many are done? It cannot tell you what happened. When results, errors, or an unknown number of tasks enter the picture, you have outgrown the counter, and the surrounding design (bounded fan-out, error collection) is covered in Worker Pools and Parallel Work with errgroup.

WaitGroup fits when you can write the number of tasks on one line before starting: Add(len(urls)). Several adjacent shapes deserve a different tool, and recognizing them keeps the counter from becoming a catch-all.

The count is discovered, not known. A crawler that finds links as it walks has no n to register. Work the count up as tasks spawn — each worker registers the children it discovers before it returns — and the WaitGroup still converges, because Add while the counter is nonzero is fine. When discovery and completion interleave arbitrarily, a channel-based worker pattern usually reads better than a growing counter.

You need the errors. A WaitGroup collects nothing: each goroutine's error evaporates unless you stash it somewhere, and stashing from multiple goroutines is its own race. The two established answers are a results channel sized to the task count, drained after Wait:

errs := make(chan error, len(urls))
wg.Add(len(urls))
for i, u := range urls {
	go func(u string) {
		defer wg.Done()
		if err := fetch(u); err != nil {
			errs <- err // buffered — never blocks
		}
	}(u)
}
wg.Wait()
close(errs)
for err := range errs {
	log.Printf("fetch failed: %v", err)
}

…or errgroup, which is WaitGroup plus error collection plus cancellation in one value.

You want progress, not just completion. Watching "12 of 40 done" wants a channel of results you can consume as they arrive; a counter only answers at the end.

The tasks must not all run at once. WaitGroup has no notion of a limit. Spawning 50,000 goroutines that each open a connection exhausts memory and file descriptors long before Wait matters — bound the fan-out with a pool of workers where the WaitGroup counts workers, not tasks:

workers := 8
wg.Add(workers)
for i := 0; i < workers; i++ {
	go func() {
		defer wg.Done()
		for u := range jobs { fetch(u) }
	}()
}
// producer closes(jobs); wg.Wait() then means all workers drained

The decision line: known count of independent tasks, only completion needed → WaitGroup. Unknown count, errors to gather, results to stream, or a limit to enforce → channels (or errgroup) — with the WaitGroup demoted to counting workers. Either way the pattern holds: register before spawning, defer Done inside, wait in the owner. What you are waiting for is unsynchronized shared state — the moment any of it is written by two goroutines, you need The Race Detector to see the damage.