Provider Mirrors for Air-gapped Networks

The corporate network has no route to registry.terraform.io, but your pipeline still has to run terraform init. Someone with connectivity prepares the providers once; every machine inside the perimeter installs from files.

Preparation happens on a connected machine, from a configuration whose constraints and lock file already say what you need:

terraform providers mirror \
  -platform=linux_amd64 \
  -platform=linux_arm64 \
  -platform=darwin_arm64 \
  /srv/terraform-mirror

The command reads the configuration's requirements, downloads the matching packages from their origin registries, verifies them, and lays them out in the packed filesystem-mirror layout — it also writes index.json and <version>.json index files, which the directory ignores when used as a filesystem mirror but which become protocol responses if you later serve the same directory over HTTP:

/srv/terraform-mirror/
└── registry.terraform.io/
    └── hashicorp/
        └── aws/
            ├── index.json
            ├── 5.31.0.json
            ├── terraform-provider-aws_5.31.0_linux_amd64.zip
            ├── terraform-provider-aws_5.31.0_linux_arm64.zip
            └── terraform-provider-aws_5.31.0_darwin_arm64.zip

Terraform accepts two filesystem-mirror layouts, and knowing both explains error messages you may see in hand-built mirrors:

The path segments are the provider's identity — registry.terraform.io/hashicorp/aws — which is why a mirror can serve providers from several different registries in one tree, and why an in-house provider is just terraform.example.com/examplecorp/ourcloud/1.0.0/linux_amd64/.

Three properties of the command decide whether the result is actually complete. -platform defaults to the machine you run it on, so a laptop-only run builds a laptop-only mirror — CI images and any ARM runners need their own -platform flags in the same invocation. Re-running against an existing directory merges: new platforms and versions are added next to the old ones, and the JSON indices are regenerated by scanning what is present, so several configurations can share one target directory and the mirror becomes the union of their requirements. And the command reads the lock file when the configuration has already been initialized, mirroring the recorded selections rather than re-picking — pass -lock-file=false only when you deliberately want the newest versions your constraints allow.

Consumption is a per-user setting, not a repository setting. Terraform reads a CLI configuration file: .terraformrc in the home directory on Linux and macOS, terraform.rc in %APPDATA% on Windows, or any file named *.tfrc you point at with TF_CLI_CONFIG_FILE:

# ~/.terraformrc
provider_installation {
  filesystem_mirror {
    path    = "/srv/terraform-mirror"
    include = ["registry.terraform.io/*/*", "terraform.example.com/*/*"]
  }
  direct {
    exclude = ["registry.terraform.io/*/*", "terraform.example.com/*/*"]
  }
}

The include/exclude pair is where teams get this wrong. Terraform tries every method whose patterns match and picks the newest available version across them, so a filesystem_mirror without a matching direct { exclude = ... } still falls through to the public registry the moment a constraint asks for a version your mirror lacks — an outage inside the firewall, or a pipeline mysteriously downloading a package you thought you controlled. Mirror-only means excluding direct for exactly the addresses you mirror.

Without any provider_installation block, Terraform still consults implied mirror directories (~/.terraform.d/plugins on Linux and macOS, %APPDATA%/terraform.d/plugins on Windows, plus a terraform.d/plugins directory beside your configuration), and automatically excludes anything found there from the implied direct method. That is convenient for a single developer, but an explicit block is what production runners should carry — the configuration is auditable, and a missing directory fails visibly instead of quietly falling back to the internet.

A filesystem mirror must be present on every machine that runs Terraform. A network mirror centralizes the same bytes behind one HTTPS endpoint, and — critically — it can serve providers whose source addresses point at registries that no longer exist on your network, because under a network mirror the hostname in the address is an identifier, not a download location.

Activation is again in the CLI configuration; the URL must use https: and should end with a trailing slash:

# ~/.terraformrc
provider_installation {
  network_mirror {
    url     = "https://terraform.corp.example/providers/"
    include = ["registry.terraform.io/*/*"]
  }
  direct {
    exclude = ["registry.terraform.io/*/*"]
  }
}

Behind that URL sits a deliberately small protocol — two GET operations per provider, implementable on any static file host:

{
  "versions": {
    "5.31.0": {},
    "5.31.1": {}
  }
}

{
  "archives": {
    "linux_amd64": {
      "url": "terraform-provider-aws_5.31.0_linux_amd64.zip",
      "hashes": ["h1:EOJImaEaVThWasdqnJjfYc6/P8N/MRAq1J7avx5ZbV4="]
    },
    "darwin_arm64": {
      "url": "terraform-provider-aws_5.31.0_darwin_arm64.zip",
      "hashes": ["h1:1xvaS5D8B8t6J6XmXxX8spo97tAzjhacjedFX1B47Fk="]
    }
  }
}

Archive URLs resolve relative to the version document, so the whole tree is plain files under one docroot. You do not hand-write those JSON files: terraform providers mirror generates them alongside the packages, and re-running it into the same directory merges new platforms and versions into the existing indices — the build step is "run the mirror command from every root we maintain", the deploy step is "upload the directory". Serving them requires the application/json media type for .json files, because a server that returns them as HTML breaks init with a parse error rather than a 404, which is one of the more confusing first-time failures.

One gap in that convenience: a provider with no origin registry cannot be fetched by the mirror command at all. Its packages must be placed into the layout by hand, and its index JSON either written by hand or avoided by running a private registry that implements the registry protocol instead of a plain mirror.

When the mirror lacks a version your constraints ask for and direct is excluded, init fails naming the provider and the constraint rather than quietly reaching the public registry. That is the behavior you want inside a controlled network: the failure surfaces at init, on a developer machine, before anything reaches a pipeline.

Two operational details decide whether this survives contact with a real organization:

• Credentials are per-hostname. If the mirror requires auth, Terraform includes credentials configured for the mirror's hostname (a credentials "terraform.corp.example" { token = ... } block, or TF_TOKEN_terraform_corp_example), and a non-standard port has to appear in that hostname (terraform.corp.example:8443). Archive downloads deliberately do not carry those credentials — a mirror that treats the zips as sensitive must hand out time-limited, signed URLs in the version document. Because the hostname becomes a pure identifier under a network mirror, one include can route several origins to the same server: ["registry.terraform.io/*/*", "terraform.example.com/*/*"].
• Trust is not transitive. A network mirror with a valid TLS certificate can serve modified provider binaries, so whoever controls that host controls your infrastructure. Lock-file checksums are the counterweight: when the mirror's packages were hashed while the config was still online, init verifies every download against them.

For in-house providers with no origin registry at all, terraform providers lock -net-mirror=URL <address> records hashes for the platforms you name, since only the mirror can vouch for those packages.

The mirror exists, terraform init works on the laptop that built it, and then the pipeline fails on the runner. Four differences between "works here" and "works everywhere" explain almost every report.

1. Hashes for platforms you never initialized on. Inside the air-gapped network nothing can consult an origin registry, so a first init there can only record checksums for the machine it runs on — leaving darwin_arm64 laptops unable to verify linux_amd64 packages. Fix it before the transfer, while connectivity still exists:

terraform providers lock \
  -platform=linux_amd64 -platform=linux_arm64 \
  -platform=darwin_arm64 -platform=windows_amd64

Pre-populating checksums for every platform your pipelines target is the step that turns a working mirror into a working team, and it must happen on a connected machine (or with -fs-mirror=/srv/terraform-mirror if the origin registry is unreachable but the mirror is present).

2. The mirror holds versions nobody's constraints accept. A mirror is a snapshot: it contains exactly what your configuration required when you ran terraform providers mirror. Widen a constraint to ~> 5.32 afterwards and init inside the firewall fails to find a matching package, because the newest thing in the mirror is 5.31.x. The mirror and the lock file must be updated as one change: widen the constraint, terraform init -upgrade and terraform providers lock while online, re-run terraform providers mirror with the same -platform list, then commit both the lock file and the mirrored tree.

Also note that terraform providers mirror reads the lock file by default — after an init, it mirrors the recorded selections rather than re-picking. Pass -lock-file=false only when you deliberately want the newest versions your constraints allow.

3. Confusing the plugin cache with a mirror. plugin_cache_dir is a download memo: it never publishes, never verifies independently, and is not concurrency-safe for parallel terraform init runs. Two rules apply — a cache directory must never also be configured as a filesystem mirror, and a cached package is only reused when its checksum already appears in .terraform.lock.hcl. Speed comes from the cache; integrity comes from the mirror and the lock file.

Updating the mirror once it is in production. Day-2 changes are a ritual rather than an accident. On a connected machine, in this order:

1. terraform init -upgrade — records the new selection and its checksums in the lock file.
2. terraform providers lock -platform=<every target> — extends hash coverage to every platform your pipelines use.
3. terraform providers mirror -platform=<every target> /srv/terraform-mirror — adds the new packages and rewrites the JSON indices in place.
4. Commit the lock file, publish the mirror directory (rsync or artifact upload), then rerun the no-network init to prove the pair still works.

Step 2 is the one teams skip: the mirror then holds bytes for every platform while the lock file holds hashes for only the machine that ran step 1, so a CI runner elsewhere fails checksum verification and the mirror gets blamed for a lock-file gap.

4. Never testing the air gap before shipping into it. The cheapest verification is a container with no network, an explicit .terraformrc, and a clean directory:

docker run --rm -v "$PWD":/tf -w /tf --network=none \
  -v /srv/terraform-mirror:/srv/terraform-mirror \
  hashicorp/terraform:1.9 init

If init completes with --network=none, the mirror layout, the include/exclude patterns, and the pre-fetched hashes are all correct. If it fails, the error names the provider it could not resolve — far friendlier than discovering the gap during a production apply.

For what happens to these same providers once they are installed and configured per region, revisit Provider Aliases for Multi-Region Use; the next stop in the roadmap is Workspace Basics with the CLI.