Debugging terraform init: 401, 403, 502, and What Your Headers Are Telling You

terraform init failed. You got one cryptic line. You don't know if the problem is authentication, network, the registry, or your configuration. The error message tells you almost nothing.

Here is how to find out what went wrong in 30 seconds using curl.

How terraform init talks to registries

Most Terraform users never see the HTTP requests that terraform init makes. Once you see the protocol, you can replay each step with curl and find the failure immediately.

The flow has three stages:

1. Service discovery. Terraform fetches /.well-known/terraform.json from the registry host. This JSON file tells Terraform where the module and provider APIs live.

2. Version listing. Terraform calls /v1/modules/{namespace}/{name}/{provider}/versions to get the list of available versions that match the version constraint in your configuration.

3. Download redirect. Terraform requests the specific version. The registry responds with a 204 No Content status and an X-Terraform-Get header pointing to the actual module archive. Terraform downloads the archive from that URL.

When terraform init fails, it could be failing at any of these three stages. The error message rarely tells you which one. Curl does.

Replace terraform init with curl

Instead of running terraform init and staring at a one-line error, replay the protocol steps with curl and read the actual HTTP responses.

Step 1: Service discovery

curl -s "https://soc2.compliance.tf/.well-known/terraform.json" | jq

Expected response:

{
  "modules.v1": "/v1/modules/"
}

If this fails, the problem is DNS, network, or the registry host itself. Nothing Terraform-specific yet.

Step 2: Version listing

curl -s "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/versions" \
  -H "Authorization: Bearer YOUR_TOKEN" | jq

Expected response:

{
  "modules": [
    {
      "versions": [
        { "version": "5.0.0" },
        { "version": "4.0.0" }
      ]
    }
  ]
}

This is how Terraform decides which version to download. It matches the version constraint in your configuration (e.g., version = "~> 5.0") against this list and picks the highest matching version. If this call returns an empty list, Terraform cannot resolve the module and fails with a "no available releases match" error.

This endpoint requires authentication. A 401 here means the same thing as on the download endpoint: missing or invalid token.

Step 3: Module download

curl -sI "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/5.0.0/download" \
  -H "Authorization: Bearer YOUR_TOKEN"

A successful response looks like:

HTTP/2 204
x-terraform-get: https://prod-ctf-registry.s3.amazonaws.com/...signed-url...
cache-control: private, no-store, max-age=0, must-revalidate
pragma: no-cache
vary: Authorization

If you see anything other than 204, the status code tells you exactly what went wrong. The rest of this guide covers each failure case.

401 Unauthorized

Symptoms

Error: Failed to query available provider packages

Could not retrieve the list of available versions for module
"terraform-aws-modules/s3-bucket/aws" from soc2.compliance.tf:
error querying module registry: 401 Unauthorized.

Causes

• No token configured. Terraform has no credentials for this registry host.
• Expired token. On compliance.tf, tokens issued by terraform login are JWTs that expire after 24 hours. Static API tokens created from the dashboard do not expire automatically but can be revoked.
• Wrong host in credentials. You have a token for registry.compliance.tf but the module source uses soc2.compliance.tf.
• Missing credentials file. The credentials.tfrc.json file doesn't exist or isn't where Terraform expects it.

Debug

Check whether Terraform can find your credentials:

# Check the credentials file exists
cat ~/.terraform.d/credentials.tfrc.json | jq

# You should see an entry like:
# {
#   "credentials": {
#     "soc2.compliance.tf": {
#       "token": "ctf_..."
#     }
#   }
# }

Verify the token is valid:

curl -sS "https://registry.compliance.tf/whoami" \
  -H "Authorization: Bearer YOUR_TOKEN" | jq

The /whoami endpoint only supports GET requests (not HEAD), so use -sS instead of -sI here. The -sS flag keeps curl silent for progress output but still shows errors if the request fails.

A valid response returns your account details, plan tier, and which frameworks you have access to. A 401 here confirms the token itself is bad.

Fix

Re-authenticate with the registry:

terraform login soc2.compliance.tf

If this is your first time setting up the registry, follow the getting started guide which covers account creation, token configuration, and your first terraform init.

Or manually create ~/.terraform.d/credentials.tfrc.json:

{
  "credentials": {
    "soc2.compliance.tf": {
      "token": "ctf_your_token_here"
    }
  }
}

For the full specification of credentials.tfrc.json, including environment variable overrides and per-host configuration, see the Terraform CLI credentials documentation.

403 Forbidden

Symptoms

Error: Failed to query available provider packages

Could not retrieve the list of available versions for module
"terraform-aws-modules/s3-bucket/aws" from soc2.compliance.tf:
error querying module registry: 403 Forbidden.

Causes

• Free tier accessing a paid framework. Your account has access to CIS but you are requesting from soc2.compliance.tf.
• Trial expired. Your trial period ended and the account reverted to the free tier.
• Module not available. The module exists in the catalog but is not yet available for the requested framework.

Debug

The error response tells you exactly what happened:

curl -s "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/5.0.0/download" \
  -H "Authorization: Bearer YOUR_TOKEN" | jq

{
  "error": "Framework 'soc_2' requires a paid subscription. Your free tier includes CIS v6.0 at cis.compliance.tf. Upgrade at compliance.tf/pricing",
  "code": "FRAMEWORK_NOT_ENTITLED",
  "request_id": "3f9c2d1e-8a4b-4c7d-9e2f-1a5b6c7d8e9f",
  "timestamp": "2026-08-20T12:00:00Z",
  "remediation": "Upgrade at compliance.tf/pricing or use cis.compliance.tf (free tier)"
}

The code field is machine-readable. The remediation field tells you what to do next, and request_id is what support will ask for if you open a ticket.

Fix

If you need SOC 2, HIPAA, or PCI DSS modules, upgrade your plan. If you are evaluating, switch to the CIS framework which is available on all tiers:

module "s3_bucket" {
  source  = "cis.compliance.tf/terraform-aws-modules/s3-bucket/aws"
  version = "5.0.0"
  bucket  = "my-data"
}

503 Service Unavailable (and the rare 502)

Symptoms

Error: Failed to query available provider packages

Could not retrieve the list of available versions for module
"terraform-aws-modules/s3-bucket/aws" from soc2.compliance.tf:
error querying module registry: 503 Service Unavailable.

Causes

• First-time module build. The specific module + version + framework combination is being generated on demand: the registry fetches the upstream module, applies compliance patches, packages the result, and uploads it to storage. Only one request wins the build; concurrent requests for the same combination wait briefly and, if the build has not landed yet, get a 503 with a Retry-After header.
• Build failure. If generation itself fails, the registry returns a 502 instead. That one does not fix itself with a retry.

Debug

# Check the response headers
curl -sI "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/5.0.0/download" \
  -H "Authorization: Bearer YOUR_TOKEN"

A 503 carries a Retry-After header that tells you how many seconds to wait before retrying, and a body with "code": "MODULE_BUILD_IN_PROGRESS". A 502 carries "code": "MODULE_GENERATION_FAILED" and a request_id to send to support.

What happens behind the scenes

The first download of any module + version + framework combination triggers an on-demand build behind a short lease. A typical cold build completes in well under a minute; while it runs, concurrent requests for the same combination return 503 with Retry-After set. Once the build lands, the patched module is cached in S3 and every later request for that combination is served directly from cache.

You usually do not have to do anything. Terraform's registry client honors Retry-After on 429 and 503 responses and sleeps for the advertised interval before retrying, so a plain terraform init normally rides out the build on its own. If you script the protocol with curl, honor Retry-After yourself:

# First attempt may return 503 while the module variant is built
terraform init

# Terraform retries automatically after the advertised delay.
# If it still fails, re-run once the Retry-After window has passed:
terraform init

A persistent 502 with MODULE_GENERATION_FAILED means the build itself failed. Retrying will not help; contact support with the request_id from the response body.

Reading response headers

A successful download response is deliberately small: four headers plus Content-Type. Here is what each one means.

curl -sI "https://soc2.compliance.tf/v1/modules/terraform-aws-modules/s3-bucket/aws/5.0.0/download" \
  -H "Authorization: Bearer YOUR_TOKEN"

HTTP/2 204
cache-control: private, no-store, max-age=0, must-revalidate
pragma: no-cache
x-terraform-get: https://prod-ctf-registry.s3.amazonaws.com/...
vary: Authorization

Where the posture hash went

Earlier versions of this post documented an X-CTF-Rules-Hash response header. It no longer exists, and neither do the other X-CTF-* headers the registry once set (X-CTF-Snapshot-Id, X-CTF-Posture-Degraded, X-CTF-Grace-Period, X-CTF-Grace-Expires). If you wrote tooling against any of them, it is reading a header that is never sent.

They were removed because Terraform never passed them on. Terraform's registry client reads exactly one response header, X-Terraform-Get, and persists none of them, so a header could not reach the tooling that needed the value.

The durable channel is the manifest written into the module archive itself: compliancetf-manifest.json, at the package root of every downloaded module (and one per declared submodule root). Read it from your Terraform working directory after terraform init:

jq .posture .terraform/modules/<module_key>/compliancetf-manifest.json

{
  "rules_hash": "aa58f1f36b906011",
  "snapshot_id": "01a0141f-be44-7065-bccf-fd10c1f1949d",
  "degraded": []
}

• rules_hash replaces X-CTF-Rules-Hash. It is the variant identity: the first 16 hex characters of a SHA-256 over the framework, the enabled and disabled controls, and the operational rules and parameters that were applied. It is also the suffix of the module version you received, so 5.15.4-aa58f1f36b906011 and this field always agree. Two environments that report the same value pulled the same transformation.
• snapshot_id replaces X-CTF-Snapshot-Id. It names the approved org posture snapshot that was applied, and is null on a framework host, where no org snapshot is involved.
• degraded replaces X-CTF-Posture-Degraded. An empty list means nothing was dropped; null means the builder did not report; a non-empty list carries the reasons.

The file is written by the build, so it travels with the artifact into your state, your CI cache, and your evidence archive. A response header did none of that.

Quick troubleshooting with curl

Two requests can rule out the most common problems before you dig deeper.

Check if the registry is reachable:

curl -s "https://registry.compliance.tf/.well-known/terraform.json" | jq

If this returns a valid JSON response, the registry is up. If it fails, the problem is DNS, network, or the registry itself.

Verify your token:

curl -sS "https://registry.compliance.tf/whoami" \
  -H "Authorization: Bearer YOUR_TOKEN" | jq

This returns your account tier, token type, expiration, and available frameworks. A 401 means the token is bad. A successful response with a different tier than you expected means your plan changed.

Quick reference

Next steps

The debugging loop is always the same: replace terraform init with a curl request, read the HTTP response, fix the problem.

• Browse the module catalog to confirm which modules and versions are available.
• Review the getting started guide for initial setup and authentication.
• Check the frameworks page to see which frameworks your plan includes.