Fuzzing with go test -fuzz

A config loader parses key=value&key2=value2 strings with DecodeKV. The table-driven suite covers normal pairs, empty input, missing values, and duplicate keys. It passes. In production, a customer's config ending in a bare & takes down the process with panic: runtime error: slice bounds out of range [:5] with length 4. The crashing input was one character away from a case somebody imagined while writing tests — and that is the whole problem: tables contain the inputs you thought of.

Fuzzing replaces imagination with a generator. The test file keeps seeds, and the engine mutates them at scale:

func FuzzDecodeKV(f *testing.F) {
	f.Add("a=1&b=2")
	f.Add("a=")
	f.Add("=orphan")
	f.Add("k=v=v")
	f.Add("")

	f.Fuzz(func(t *testing.T, raw string) {
		m, err := DecodeKV(raw)
		if err != nil {
			return // rejected input: surviving it without panicking already counts
		}
		out := EncodeKV(m)
		back, err := DecodeKV(out)
		if err != nil {
			t.Fatalf("re-encoding %q produced undecodable output %q: %v", raw, out, err)
		}
		if len(back) != len(m) {
			t.Fatalf("round trip changed size: %#v -> %#v", m, back)
		}
		for k, v := range m {
			if back[k] != v {
				t.Errorf("round trip altered %q: %q -> %q", k, v, back[k])
			}
		}
	})
}

The signature is the contract: f *testing.F for the setup phase, f.Add to register seed inputs, and f.Fuzz(func(t *testing.T, raw string) {...}) — one function taking the generated values, typed from the closure's parameters. Strings and []byte are the usual choices; integers, floats, and bools also work, which is how you fuzz a parser that takes (data []byte, offset int).

Two properties make this cheap to adopt. Every f.Add seed runs as a normal test on every plain go test ./..., so the seeds are regression cases with zero extra machinery. And the engine does not need you to name the bug: "does not panic" is enforced implicitly by the framework, while the round-trip block asserts the deeper property — encode then decode must preserve every pair. A function that silently drops a key on weird input passes any "no crash" test and fails this one.

Running it:

go test ./parser                              # seeds and corpus only, fast
go test -fuzz=FuzzDecodeKV -fuzztime=30s ./parser

-fuzz is a regex matched against fuzz target names, and it starts the mutation engine: it takes the seeds, flips bytes, splices them, appends prefixes, and feeds hundreds of thousands of variants per second. -fuzztime=30s bounds the run — without it, fuzzing runs until you stop it. Within the first minute the engine finds the & suffix, minimizes toward the smallest input that still fails, and prints the reproduction.

The engine found the bug and printed a failing input. Deleting that output after fixing the panic returns the codebase to the exact state where the bug shipped: nothing ensures the case stays covered while the parser is refactored three months later.

Go's answer is the corpus directory. On failure, the engine writes the input to a file:

testdata/fuzz/FuzzDecodeKV/4f1a9c2e8b70d315
go test v1
[]byte("a=1&")

Commit it. From then on, go test ./parser replays every file in testdata/fuzz on every run — no -fuzz flag, no engine, milliseconds of cost — and if a future change reintroduces the panic, the run fails with that exact input. The crasher became a permanent regression test that nobody had to write by hand. To re-run a single entry, match it by name: go test -run 'FuzzDecodeKV/4f1a9c2e8b70d315' ./parser.

The corpus also works in the other direction: add inputs worth keeping as seeds with f.Add, or drop files into testdata/fuzz by hand. Curating seeds is the cheapest way to steer the engine — it mutates what you give it, so realistic config strings, unicode keys, control characters, and strings near length boundaries become the neighborhood it explores.

Two operational facts shape where fuzzing belongs in the pipeline.

The engine is expensive and runs one target at a time. -fuzz executes a single fuzz target per package invocation (Go deliberately prevents the engine from running targets concurrently against the same package), and it saturates CPU for as long as it runs. It does not belong in the per-commit job. The working split: plain go test ./... — seeds plus corpus replay — on every commit; a nightly or scheduled job with -fuzztime=10m (longer for parsers handling untrusted input) that opens a ticket with any new corpus file.

A fuzz failure must be reproducible to be actionable. The engine minimizes the input and reports it so you can run the target against it directly. If the same input passes on re-run, the target is nondeterministic — it read the wall clock, iterated a map in an order it asserted on, or spawned goroutines whose scheduling decided the assertion. Determinism is the next topic's concern, because it decides whether a found failure is a bug report or a flake.

Two teams adopt fuzzing the same week. One finds a heap overflow in four minutes; the other's engine runs a million iterations and finds nothing while their parser stays shallow. The difference is the target, and the craft comes down to invariants, speed, and determinism.

An invariant beats an absence. t.Error("should not panic") never appears — not panicking is automatic. What you add is a property the correct function guarantees, in order of how much they catch:

• Round trip: Decode(Encode(x)) equals x, as in the target above. Catches silent corruption, not just crashes.
• Idempotence: Normalize(Normalize(s)) == Normalize(s). A function meant to stabilize input should not oscillate between two outputs.
• Equivalence with a reference: a slow, obviously correct implementation agrees with the optimized one on every generated input. The optimizer is free; the reference is trusted.
• Domain bounds: a parsed size is never negative, a decoded UTF-8 string contains no replacement runes the input did not have.

Each invariant converts "the fuzzer produced weird bytes" into a specific accusation. Without one, the engine can only report panics — valuable, but it will happily accept a function that returns a plausible-looking wrong answer forever.

Speed is the budget. The engine's yield is executions per second; a target doing network calls or database work runs thousands of iterations instead of millions and explores proportionally less. Keep the target in pure CPU territory — if it needs a fixture, build it once before f.Fuzz, not inside. This is also why fuzzing complements benchmarks: the benchmark tells you the function is hot, the fuzz target hammers it with inputs nobody types.

Determinism is non-negotiable. A target that consults time.Now() or wall-clock-dependent code can fail at midnight and pass at noon on identical input, which makes minimization meaningless. Inject a fixed clock — the same discipline time handling argues for in production code. Avoid asserting on map iteration order: range over a map to build a set, then compare sets.

Seed deliberately. The engine explores the neighborhood of your seeds, so include each character class the grammar allows: empty string, single character, delimiters at the start and end, repeated delimiters, multi-byte runes, and one input of unusual length. Seeds are also the readable half of the suite — the fuzz engine handles the rest of the grammar while the seeds document it.