Worker Pools

An image service receives 10,000 upload events and the obvious code passes review:

for _, img := range images {
	wg.Add(1)
	go func(img Image) {
		defer wg.Done()
		results <- thumbnail(img) // one goroutine per image
	}(img)
}

It works in staging with 50 images. In production the burst arrives at once: 10,000 goroutines materialize, each holding its own image bytes in memory, each competing for four cores through the scheduler, and the decoder's file handles multiply until the process is either thrashing or out of file descriptors. Goroutines are cheap — a few kilobytes each — but 10,000 of them is not a concurrency strategy, it is an absence of one. The service does not want 10,000 things happening; it wants four things happening, most of the time.

A worker pool fixes the count. A fixed number of long-lived goroutines share one jobs channel; the channel — not the code — controls how much work exists at once:

func pool(images <-chan Image, n int) <-chan Result {
	results := make(chan Result, n)

	var wg sync.WaitGroup
	for i := 0; i < n; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for img := range images {
				results <- convert(img)
			}
		}()
	}

	go func() {
		wg.Wait()
		close(results)
	}()
	return results
}

Read what changed structurally. Goroutine count is now n, set before launch, independent of job count — 10 or 10,000,000 jobs produce exactly n workers. Each worker loops over the shared jobs channel, which means idle workers are parked in a receive, costing nothing, and a burst of jobs is processed n at a time with the rest waiting in the channel (or at the producer, if the buffer is full — this is backpressure from a buffered channel, working as designed). The results channel's buffer of n guarantees a worker can always deposit its result without a rendezvous with the consumer, keeping workers from queueing on the drain path.

The closing pattern is the one from multi-producer close: workers never close results — two finishing workers would double-close and panic. Instead a detached goroutine waits on the WaitGroup and closes exactly once after the last worker's range ends. The whole construction uses only primitives already covered: directional signatures (images <-chan Image) declare which side owns each stream, range terminates workers when the job source closes, and the coordinator owns the shutdown of the output.

The pretty version hides three production questions: what happens when jobs arrive faster than the pool drains them, what happens to results nobody reads, and what happens when a job fails. Here is the pool with all three answered:

type Job struct {
	ID   int
	Path string
}

type Result struct {
	ID       int
	ThumbURL string
	Err      error
}

func RunPool(jobs <-chan Job, workers int, buf int) <-chan Result {
	results := make(chan Result, buf)

	var wg sync.WaitGroup
	for i := 0; i < workers; i++ {
		wg.Add(1)
		go func(id int) {
			defer wg.Done()
			for j := range jobs {
				url, err := makeThumb(j.Path)
				results <- Result{ID: j.ID, ThumbURL: url, Err: err} // errors ride the result channel
			}
		}(i)
	}

	go func() {
		wg.Wait()
		close(results)
	}()

	return results
}

func main() {
	jobs := make(chan Job, 256)

	go func() {
		defer close(jobs)
		for _, j := range scanQueue() {
			jobs <- j // blocks at 256 deep while the pool is saturated: backpressure, not memory growth
		}
	}()

	failed := 0
	for r := range RunPool(jobs, 8, 64) {
		if r.Err != nil {
			failed++
			log.Printf("job %d: %v", r.ID, r.Err)
		}
	}
	log.Printf("done, %d failed", failed)
}

Each design decision has a failure it prevents. Errors ride the results channel as data rather than being logged inside workers: the consumer already ranges over results, so failure handling lives in exactly one place, and a worker that returned early on first error would have silently skipped jobs — instead every job produces exactly one result, success or failure. This mirrors the errors-as-values discipline from Error Handling, carried across a channel boundary. The producer's jobs <- j blocks at buffer depth rather than allocating a million-job slice: when the pool saturates, the backlog stops at 256 and the producer (maybe an HTTP handler) applies natural backpressure upstream. The consumer's range over results terminates because the coordinator closed it after all workers exited — which in turn happened because someone closed jobs. Chain the closes correctly and shutdown is automatic; break any link and you get a goroutine leak, the subject of Goroutine Leaks.

What this shape deliberately does not do: cancel early. If the consumer stops reading after the first error, workers finish their current job and then block forever sending results nobody takes — a leak waiting to happen. Cancellation needs a done signal threaded through every send, which is where context plugs into this exact structure in the next topics.

How many workers? The answer is not runtime.NumCPU() by reflex — it depends on what the worker spends its time doing.

CPU-bound work (encode, compress, hash, compile): each worker occupies a core for the whole job, so more workers than cores means contention and context-switch overhead for no gain. Start at runtime.GOMAXPROCS(0) — the parallelism the Go scheduler itself targets — and raise only with profiling evidence:

workers := runtime.GOMAXPROCS(0) // CPU-bound default

I/O-bound work (HTTP calls, database queries, disk reads): the worker spends most of its life waiting, and a waiting worker holds no core. The limit is the downstream system's tolerance — connection-pool size, API rate limits, ulimit — not your CPU count. An HTTP fan-out to a service that tolerates 32 concurrent requests wants ~32 workers, and the honest way to find the number is to drive it up and watch error rates and p99 latency at the dependency:

workers := downstreamMaxConcurrency // e.g. 32, from the API's published limit

Mixed workloads usually want two pools or a weighted queue — one slow-blocking class will otherwise idle the slots that fast work could use. A quick diagnostic for mis-sizing: if workers sit in runtime.BlockingProfile-style waits while the jobs queue is deep, you are I/O-starved and should add workers; if the run queue is saturated and wall-clock stops improving, you are CPU-bound and workers are overhead.

The measurement loop matters more than the initial guess — instrument queue depth and per-job latency, then adjust:

Sizing also decides failure character. Too few workers and the system degrades into latency (queues grow, requests time out); too many and it degrades into errors at whatever resource you oversubscribed — memory, sockets, the dependency's own limits. Interviewers ask for the number you would pick and the reason it is a measurement, not a constant; the answer that lands is GOMAXPROCS for CPU work, the dependency's concurrency budget for I/O work, and a plan to watch queue depth to revise both.