Composing Modules into Layers

A team hands you a 4,000-line main.tf that provisions three environments by copy-paste. The refactor is not "extract some modules" — it is answering one question per block: does this value describe how to build the thing, or which thing this environment builds? The inputs and outputs you designed in Designing Module Input and Output Contracts are the joints; this topic decides where the bones sit.

The answer becomes the directory layout:

repo/
├── modules/                     # how to build things (environment-blind)
│   ├── network/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── outputs.tf
│   ├── database/
│   └── observability/
└── envs/                        # which environment this is
    ├── dev/
    │   ├── main.tf              # module calls + wiring only
    │   ├── backend.tf           # state location for dev
    │   └── terraform.tfvars     # account id, region, sizing
    ├── staging/
    └── prod/

Each side holds artifacts the other must never contain:

Two failure modes show up constantly. The first is secrets and identity leaking downward: an account ID or region hardcoded inside modules/network means the module only works for one environment, and the "shared" copy silently forks. Push it up as an input — if it must differ, it is a root's job. The second is provider configuration leaking upward into the child: a provider block inside a shared module forces every caller to hand it configurations through the providers meta-argument and forbids count/for_each on that call entirely. Providers configure in the root; modules consume the default (or an explicitly passed) one.

The layout also decides your state story. Three environment roots mean three states, three locks, three backends — a change to envs/dev cannot corrupt prod because prod's state is a different object, and a CI pipeline can plan only the directory the branch touched. Compare that with three workspaces sharing one configuration: cheaper to set up, impossible to give different backends, and one mistaken terraform workspace select applies prod code to staging. The deeper treatment is in Promoting Changes Across Environments; the point here is that layering is what makes environment isolation a filesystem property instead of a runbook step.

What a compliant root actually looks like is shorter than people expect:

# envs/prod/main.tf — composition and wiring, nothing else
terraform {
  backend "s3" {
    bucket         = "acme-tfstate-prod"
    key            = "prod/network/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "acme-tf-locks"
  }
}

provider "aws" {
  region = var.region
}

module "network" {
  source = "../../modules/network"

  name     = "acme-prod"
  cidr     = var.vpc_cidr
  az_count = 3
}

output "vpc_id" {
  description = "VPC id for downstream deployments."
  value       = module.network.vpc_id
}

No resource blocks. Every line either selects the environment or composes a module — if you need a resource here, ask whether the shared module missed one.

Interviewers ask for this structure cold: "show me where the prod backend config goes, where the VPC resources go, where the account ID goes." Answer with the three paths above and you have demonstrated the whole model.

Composition tempts you toward depth: app calls network calls subnets calls tags. Each level is locally reasonable and the result is a tree nobody can read a plan out of.

module.app.module.network.module.subnets.aws_subnet.private[2]

That address is the symptom. Everything downstream of it — plan diffs, terraform state, -target operations, incident debugging at 2 a.m. — inherits its length. The rule that keeps a configuration operable: two levels from the root. Root calls leaf modules; a leaf may call one helper; nothing nests further.

Flat, remember, does not mean "one module does everything." A flat root typically calls six to ten thin modules, one concern each:

module "network"        ← VPC, subnets, route tables
module "database"       ← RDS instance, parameter group
module "observability"  ← log groups, alarms, dashboard
module "dns"            ← zones and records

Each leaf owns one kind of resource, exposes a small contract, and knows nothing about its siblings — the composition (who calls whom, in what order) is decided entirely at the root, where you can read it in one screen. Depth hides that decision inside child files; flatness keeps it where the reviewer already is.

Depth does buy encapsulation — a caller of app need not know about subnets — but Terraform already gives you a better version of that trade: a wrapper module keeps the nesting out of your plan while keeping the interface small. The wrapper is an ordinary module that calls several leaves and fixes their defaults:

# modules/app/main.tf — one level from the root, three leaves inside
module "network" {
  source = "../network"

  name   = var.name
  cidr   = var.cidr
  azs    = var.availability_zones
}

module "database" {
  source = "../database"

  vpc_id     = module.network.vpc_id
  subnet_ids = values(module.network.private_subnet_ids)
  engine     = "postgres"
  multi_az   = var.environment == "prod"
}

output "db_endpoint" {
  description = "Connection endpoint of the provisioned database."
  value       = module.database.endpoint
}

From the root's perspective this is still one level: module.app.db_endpoint. Inside, the leaves are wired once, with the awkward parts (engine = "postgres", the prod multi-az rule) fixed in exactly one place instead of restated by every caller — which is the wrapper's whole justification. Do not read it as "nesting is fine now": the wrapper's internals stay flat and visible when you debug them.

When a plan surprises you, the flat tree makes the tooling usable:

terraform state list | grep '^module\.'   # every instance address, sorted by layer
terraform state show 'module.database.aws_db_instance.primary'
terraform graph -draw-cycles               # who depends on whom, one hop at a time

Plan: 3 to add, 1 to change, 0 to destroy.
  ~ module.database.aws_db_instance.primary
      + instance_class: "db.t3.large" → "db.t4g.large"

With a deep tree the same diff appears under module.app.module.database... three times, and you must confirm which of the duplicated module paths you are even looking at. Readability of the plan is not a cosmetic concern — it is the control surface for production changes, and every nesting level you add taxes it forever.

Here is a root composing three leaves, complete and unremarkable — which is the point:

module "network" {
  source = "../../modules/network"

  name       = "acme-${var.environment}"
  cidr       = var.vpc_cidr
  az_count   = 3
}

module "database" {
  source = "../../modules/database"

  vpc_id      = module.network.vpc_id
  subnet_ids  = values(module.network.private_subnet_ids)
  kms_key_arn = module.kms.key_arn
}

module "observability" {
  source = "../../modules/observability"

  cluster_name = "acme-${var.environment}"
  target_arns  = [module.database.instance_arn]
}

Read the wiring: every cross-module dependency is an output of one module bound to an input of another. That single rule is what keeps the dependency graph honest. Terraform infers ordering from the value flow, the plan shows a real edge instead of a hidden one, and you can delete module.network to see exactly which consumers break — they are the ones whose inputs went unbound.

Now the shortcuts teams take instead, and why each is worse:

The wrapper from the previous section is where these rules consolidate. Its inputs are few (name, cidr, availability_zones, environment), its internal wiring is fixed, and its outputs are exactly what callers consume. Many teams then instantiate it once per service:

module "service" {
  source   = "../../modules/service"
  for_each = var.services

  name        = each.key
  environment = var.environment
  cidr        = each.value.cidr
}

Each module.service["payments"] is an independent instance with its own state subtree — the composition scales by data, not by copy-paste.

Trying to bypass the contract fails fast, and the error teaches the model:

Error: Unsupported attribute

This object does not have an attribute named "aws_vpc".

That is what you get for writing module.network.aws_vpc.this.id from a sibling. The module.network object exposes exactly one class of attribute — its outputs — so Terraform rejects the resource address at expression evaluation, before any plan output is produced. There is no "reach in and grab it" mode, and configurations that appear to do so are always reading an output someone happened to name like a resource.

Remote state is the sneakier shortcut because it works — until it does not. It holds while both sides are healthy and nobody moves the sharing convention. It breaks when the network state changes key, when a reader runs before the writer has applied, or when a reader takes the writer's state lock and stalls the other environment's deploy. And because terraform_remote_state returns a plain object with no schema validation, a renamed output surfaces as an Unsupported attribute in a different pipeline, days later, with no link back to the change that caused it.

Be honest about the wrapper's failure mode: when every caller must override half of its fixed defaults, the wrapper has become a pass-through with extra steps, and the defaults it "fixes" were never real decisions. Fix it by either committing to the opinion (fewer inputs) or dissolving the wrapper and exposing the leaves directly. There is no third option that stays maintainable.

This is also where module distribution starts to matter: once five teams compose these same leaves, hand-copied paths stop working, and the next topic — Publishing to a Private Module Registry — is how the composition becomes a shared, versioned dependency instead of a folder everyone clones.