Configuration and Secrets Handling

A service starts, accepts traffic, and fails the first time it touches the database — because DATABASE_URL was never set and the code read it into an empty string. The failure surfaces as a connection error in a request log, six hours after the deploy, with nothing in the message naming the actual problem: a missing configuration key. The shape of this incident is always the same — configuration is read lazily, from everywhere, and validated never.

The alternative is a single load step at startup that builds one typed struct, validates everything, and refuses to start if anything is wrong.

package config

type Config struct {
	DatabaseURL string
	ListenAddr  string
	LogJSON     bool
	ShutdownGrace time.Duration
}

func Load() (Config, error) {
	cfg := Config{
		ListenAddr:      ":8080",                  // defaults live next to the field
		LogJSON:         true,
		ShutdownGrace:   30 * time.Second,
	}

	var errs []string
	if v, ok := os.LookupEnv("DATABASE_URL"); ok {
		cfg.DatabaseURL = v
	}
	if v, ok := os.LookupEnv("LISTEN_ADDR"); ok {
		cfg.ListenAddr = v
	}
	if v, ok := os.LookupEnv("LOG_JSON"); ok {
		b, err := strconv.ParseBool(v)
		if err != nil {
			errs = append(errs, "LOG_JSON: "+err.Error())
		} else {
			cfg.LogJSON = b
		}
	}
	if v, ok := os.LookupEnv("SHUTDOWN_GRACE"); ok {
		d, err := time.ParseDuration(v)
		if err != nil {
			errs = append(errs, "SHUTDOWN_GRACE: "+err.Error())
		} else {
			cfg.ShutdownGrace = d
		}
	}

	if cfg.DatabaseURL == "" {
		errs = append(errs, "DATABASE_URL is required")
	}
	if len(errs) > 0 {
		return Config{}, fmt.Errorf("invalid configuration:\n  - %s", strings.Join(errs, "\n  - "))
	}
	return cfg, nil
}

Three properties make this work in practice. Fail fast: main calls config.Load() before anything else, so a missing key kills the deploy, not the request. Accumulate errors: reporting only the first problem would produce a five-deploy debugging marathon; one message listing every bad field lets operations fix the environment in a single pass. Typed fields: ShutdownGrace is a time.Duration, not a string, so no other package ever sees raw text — configuration is parsed once, at the boundary.

Where the struct travels matters as much as how it is built. Pass config.Config as an explicit argument to constructors rather than exporting a package-level singleton:

srv, err := server.New(cfg)   // tests pass Config{...} directly

A global config.Get() makes every test mutate process-wide state and hides dependencies; an explicit parameter keeps the wiring visible and lets a test supply an in-memory config with a fake URL. This is the constructor discipline from Idiomatic API and Package Design applied to configuration.

The environment is the right source for values that vary per deployment — ports, URLs, credentials injected by the orchestrator — because it keeps the artifact identical across environments. Code, defaults, and validation stay in the repository; only the values differ. For values that must not vary that way — secrets — the environment is the second choice, and the next chapter explains what beats it.

A service half-uses the pattern above: os.Getenv calls are spread across six packages, a Redis client reads REDIS_ADDR directly, and one package reads it as REDIS_HOST. The environment has REDIS_ADDR set; the client silently falls back to localhost:6379 and connects to the wrong place in exactly one environment. Configuration sprawl does not fail loudly — each Getenv carries its own default, its own fallback, and its own spelling.

The structural fix: configuration enters the program in one place, in one struct, with one precedence rule.

func load(fs *flag.FlagSet) (Config, error) {
	cfg := Config{}

	// Layer 1: defaults written in Go.
	cfg.RedisAddr = "localhost:6379"

	// Layer 2: environment.
	if v := os.Getenv("REDIS_ADDR"); v != "" {
		cfg.RedisAddr = v
	}

	// Layer 3: explicit flags, if the binary has them (CLI beats env).
	if f := fs.Lookup("redis-addr"); f != nil {
		set := false
		fs.Visit(func(fl *flag.Flag) {
			if fl.Name == "redis-addr" {
				set = true
			}
		})
		if set {
			cfg.RedisAddr = f.Value.String()
		}
	}
	return cfg, validate(cfg)
}

The exact mechanics of detecting a present flag live in Command-Line Flags with the flag Package; the point here is architectural: the layering appears once, in config, and every other package receives values already resolved. Precedence becomes a documented sentence — flags beat environment beat defaults — instead of emergent behavior.

Validation belongs to the same step, not to first use. validate(cfg) checks cross-field and semantic constraints before anything starts:

func validate(c Config) error {
	var errs []string
	u, err := url.Parse(c.DatabaseURL)
	if err != nil || u.Scheme == "" {
		errs = append(errs, "DATABASE_URL must be an absolute URL")
	}
	if c.ShutdownGrace < time.Second {
		errs = append(errs, "SHUTDOWN_GRACE must be at least 1s")
	}
	switch c.LogLevel {
	case "debug", "info", "warn", "error":
	default:
		errs = append(errs, "LOG_LEVEL must be debug|info|warn|error")
	}
	if len(errs) > 0 {
		return errors.New(strings.Join(errs, "; "))
	}
	return nil
}

Parsing a duration or a URL at load time converts an environment mistake into a startup error with the field name attached. The failure the original incident needed — SHUTDOWN_GRACE: time: invalid duration printed during deploy — is this function's output.

Two framework temptations deserve explicit judgment. Reflection-based libraries that map struct tags to environment names remove twenty lines and add a dependency, an implicit naming convention, and error messages you do not control; they are reasonable when a service truly has fifty fields, and premature below that. And struct tags have a trap of their own: tags are visible via reflection to anything that serializes the struct — a json:"-" or careful field export discipline keeps a Secret field out of API responses even if someone later adds json.Marshal(cfg). Structure the load, keep the raw environment out of business logic, and every consumer of configuration inherits the validation for free.

The API key lives in the deployment environment. Rotation means editing the orchestrator's config and restarting the pod — a restart that drops in-flight requests, requires coordination, and happens once a quarter because it hurts. Meanwhile an engineer debugging the service prints the config struct to stdout "just to check the values," and the key is now in the log aggregator with a 30-day retention policy.

Each of those has an established fix, and they are independent.

Prefer file-mounted secrets. Docker and Kubernetes both render secrets as files (/run/secrets/db_password), and a file beats an environment variable for three reasons: environment variables are visible to every child process and through /proc/<pid>/environ, they cannot be updated in place, and they end up in docker inspect output. A file can be mounted read-only, mode 0600, and replaced atomically without touching the process:

func readSecret(path string) (string, error) {
	b, err := os.ReadFile(path)
	if err != nil {
		return "", fmt.Errorf("read secret %s: %w", filepath.Base(path), err)
	}
	return strings.TrimSpace(string(b)), nil
}

// usage: read on use, not once at startup
password, err := readSecret("/run/secrets/db_password")

Reading on use (or with a short cached TTL) is what makes rotation work: the orchestrator swaps the file contents, the next connection picks up the new value, and no restart occurs. For long-lived clients, a SIGHUP handler that re-reads the secret directory gives operators rotation on demand — the same signal-driven pattern Graceful Shutdown and Connection Drain uses for termination.

Redact at the log boundary. Structured logging makes leaks easy: one slog.Any("config", cfg) in a startup log publishes every field, secrets included. The defense is threefold — never log the config struct as a whole; mark sensitive types with a String() method that prints [REDACTED] so even a careless %v stays safe; and keep secrets out of error strings, which travel into log records through the error chain:

type Secret string

func (s Secret) String() string   { return "[REDACTED]" }
func (s Secret) GoString() string { return "[REDACTED]" } // %#v too
func (s Secret) MarshalJSON() ([]byte, error) {
	return []byte(`"[REDACTED]"`), nil
}

The MarshalJSON method is the last gate before an API response or a debug dump — it makes accidentally serializing the struct harmless.

Keep the artifact clean. Committed .env files, keys in test fixtures, and — circling back to Embedding Assets with go:embed — //go:embed .env all bake credentials into the repository and the binary, where they are copied forever. .gitignore plus a CI check that greps tracked files for known key patterns (AKIA, -----BEGIN ... PRIVATE KEY) catches the common cases; the structural rule catches the rest: if a value would be embarrassing in a pull request, it never enters the repository, and the binary contains code while the environment contains identity. Configuration decides behavior; secrets decide access — loading them from files, redacting them from output, and rotating them without restarts keeps all three of those statements true in production.