Workspace Design Tradeoffs

Choosing workspaces is choosing to share. Every .tf file, the .terraform.lock.hcl, the backend block, provider constraints, and module versions are read identically whether you are standing in dev or production. Only two things vary: inputs you pass at runtime, and branches written against terraform.workspace. Before committing to that model, look at exactly what stops varying.

Variable files are the first surprise. terraform.tfvars and *.auto.tfvars are loaded by path, not by workspace, so an account_id written there applies to every workspace in the directory. Per-environment values must be selected by the caller:

terraform workspace select staging
terraform plan -var-file=environments/staging.tfvars -out=tfplan

Keep one .tfvars file per environment, name it after the environment, and make the pipeline pass it explicitly. A terraform.tfvars that "temporarily" hard-codes one account is the classic way staging starts writing into production's account.

What the shared configuration implies, dimension by dimension:

Two rows deserve a second look. Module versions cannot differ. To canary a module — ~> 2.4 in staging while production stays on ~> 2.1 — you must drive the version argument from a variable, which means the configuration no longer pins what production runs; you traded the canary for the pin. And prod-only resources cost conditionals. Every WAF, every extra replica set, every KMS key that exists in exactly one tier becomes another terraform.workspace == ... branch. Five branches are fine; twenty make the root a conditional matrix where deleting staging requires re-reading every resource block.

None of this is news if you have been running terraform workspace show for a while — Workspace Basics with the CLI covered how little a workspace actually changes. A short discipline list makes the shared model survivable:

1. One .tfvars file per environment, named after it and passed explicitly — never environment-specific values committed to shared files.
2. Assert the pairing in CI: test "$(terraform workspace show)" = "$VARFILE_SUFFIX" so a staging plan cannot silently run with production's values.
3. Treat .terraform.lock.hcl as global: one provider resolution for every tier, so a provider bump lands everywhere in the same merge.

Providers deserve the third point spelled out, because the lock file being shared means you cannot run provider 5.31 in staging while production stays on 5.29 — the canary many teams rely on before adopting a release with breaking schema changes. Under this model that canary happens in a branch, not in an environment.

This is the honest trade: workspaces buy you identical structure — which is genuinely valuable when environments are supposed to be clones differing only in size — and charge you in flexibility the moment they are not. The counter-model duplicates wiring instead of logic: separate directories whose only job is to hold a backend block, a variable file, and a list of module calls, with the real logic living in shared modules. That model's costs (drift between copies, more directories in CI) and its payoff (divergent providers, credentials, and module versions) are the subject of Directory Layouts per Environment.

Here is the scenario interviewers like, because it has happened to everyone: you are updating staging. Your shell is in the right directory, the diff looks like the two-instance staging stack, you run terraform apply, and it reaches production — because an hour ago you ran terraform workspace select production to inspect something and never switched back.

Nothing about the plan shouted. The diff for one workspace is structurally identical to the diff for another — same resource addresses, same module tree, same lines — because both are generated from the same configuration. The workspace name lives only in .terraform/environment, in TF_WORKSPACE if you exported it, and in your memory. Three concrete ways this class of mistake happens:

1. Stale selection. The workspace is per directory, not per shell, so switching in one terminal switches it everywhere, including CI jobs that reuse a checkout.
2. Typo'd workspace names. terraform workspace new prodction happily creates a fresh, empty workspace. The next plan proposes to create the whole estate; applying it in a shared account fails halfway and leaves orphans.
3. Forgotten TF_WORKSPACE. The CLI docs explicitly warn that in a local shell "it can be easy to forget the variable is set and apply changes to the wrong state."

Defence in depth, in the order that actually works:

set -euo pipefail
CURRENT="$(terraform workspace show)"
if [ "$CURRENT" != "$EXPECTED" ]; then
  echo "wrong workspace: expected $EXPECTED, got $CURRENT" >&2
  exit 1
fi
terraform plan -input=false -var-file="$EXPECTED.tfvars" -out=tfplan

Run that assertion in every pipeline before init's results are used, and echo terraform workspace show into the job log so the record of every run names the workspace. In configuration, add a precondition guard that fails the plan when terraform.workspace disagrees with the environment variable the pipeline injected — a failing precondition aborts the run, whereas a check block only warns. Tag every resource with Environment = terraform.workspace so the cloud console identifies each object after the fact.

Two mitigations are weaker than they look. The plan file alone does not identify its workspace — the state key prefix does, in the backend, not in the diff. And -target does not make a run safer; it changes which resources are considered, not which state is written.

5. Destroy from the wrong workspace. Contained — it can only destroy what that workspace tracks — but it is the same mistake wearing a louder costume, and the precondition guard catches it for the same reason.

When an apply lands in the wrong tier, recovery is mechanical, and knowing the sequence keeps it calm:

1. Stop. Do not try to undo it with a second apply from the wrong directory.
2. Select the correct workspace and run terraform plan: the inverse diff shows exactly what the wrong apply changed, expressed as the changes needed to return.
3. Apply that diff to restore the tier, then run the intended change from the right workspace.
4. Pull a state backup first — terraform state pull > backup.json — and check the backend's version history (S3 object versioning) if entries were deleted.

Tagging turns detection from guesswork into a query. With Environment = terraform.workspace on every resource, aws ec2 describe-instances --filters Name=tag:Environment,Values=production tells you in seconds whether an object belongs to the tier you believe it does — and a Environment=staging tag sitting in the production account is the alarm you want to see early.

The residual risk is structural: everyone who can run Terraform against this backend can select every workspace, including production, with the same credentials. Workspace boundaries are labels on state, not permissions. When that stops being acceptable — separate accounts, restricted state access, prod-only reviewers — you have outgrown workspaces for permanent tiers, and the decision framework in the next chapter, Directory Layouts per Environment, is how you formalise the split.

Workspaces solve a specific problem: many states for one identical configuration. The tool is wrong the moment your environments are not identical, or the moment the states must be governed differently. Use this decision table rather than team habit:

Three of those rows are structural, not stylistic.

Before deciding, ask five questions of the estate:

1. Can a developer holding staging credentials read production state today? If yes, directories over separate buckets are not optional.
2. Does production own resources the other tiers do not? If yes, conditionals will accumulate until the root is a matrix of branches.
3. Must production run an older provider or module release for stability or compliance? If yes, you need per-tier pins.
4. Should a merge touching only dev be unable to change production's files? Directories give you that structurally; workspaces cannot.
5. How many environments exist and how long do they live? A dozen pull-request previews dying within 48 hours belong in workspaces; three permanent tiers do not.

State credentials. One backend "s3" block means one credential path for every workspace. With directories, envs/prod/backend.tf points at a bucket that only the production deploy role can touch — staging developers lose s3:GetObject on production state outright. No workspace setting can produce that separation.

Merge blast radius. Under workspaces, every merge lands on the configuration that all tiers read; whether it applies is per workspace, but the code path is shared. With directories, a change confined to envs/dev/** physically cannot modify what production's root contains, and CI can skip production entirely.

Version lag. Each root pins its own module version and its own .terraform.lock.hcl, so staging can adopt module ~> 2.4 while production waits a week on ~> 2.1. Under a shared configuration there is only one version constraint to write.

The classic objection to directories — "now we copy the same 80 lines three times" — has a real answer, and it is tooling rather than discipline. Terragrunt keeps one definition and generates the glue per directory:

# envs/staging/vpc/terragrunt.hcl
include "root" { path = find_in_parent_folders() }

inputs = merge(
  read_terragrunt_config(find_in_parent_folders("common.hcl")).inputs,
  { node_count = 3 }
)

The root terragrunt.hcl generates the backend and provider blocks and reads shared inputs once, so each environment directory is ten lines while state stays fully separate. Duplicated logic is the smell; duplicated wiring is cheap. HCP Terraform Stacks attacks the same problem from the platform side: one stack definition, one stack instance per environment, each instance holding its own state.

Teams often end up running both mechanisms, and that is a coherent answer rather than indecision: directories for the permanent tiers — dev, staging, production — plus workspaces inside the dev root for per-pull-request previews, where clone-and-throwaway is exactly the behaviour you want (terraform workspace new pr-482, apply, terraform workspace delete). The two stop competing once you see they answer different questions: directories separate governance, workspaces separate copies.

So the pragmatic rule most teams land on: workspaces for disposable environments that should be clones, directories for permanent tiers that carry different blast radius, credentials, and owners. The next chapter builds the directory model concretely — see Directory Layouts per Environment.