blinqas/terraform-station-test

Terraform Station Test Skill

Use this skill for testing this repository's root Terraform module (Station) with Terraform's built-in test framework.

Station is consumed as a module. Tests must validate behavior through module inputs and outputs, not by treating the root as a standalone deployment.

What this skill covers

• Writing new Station test files in tests/*.tftest.hcl
• Updating existing feature tests (application, group, tfe, connectivity, identity, user_assigned_identities)
• Correct use of setup modules in tests/setup-*/
• Correct variable override patterns using prior run outputs
• Running tests locally with terraform test / terraform test -filter=...
• Aligning test changes with CI selective test execution

Station test ground rules

1. Always inspect the module inputs before writing assertions:
   • variables.tf
   • variables.applications.tf
   • variables.identity.tf
   • relevant child-module variable files when needed
2. Prefer expanding an existing run block in a test file instead of creating new blocks unless logic is meaningfully different.
3. The first run blocks in Station tests should be setup/bootstrap runs (for dynamic IDs/resources needed by main runs).
4. Use setup outputs to override placeholder values in the main test variables block(s).
5. Validate both minimum and maximum configurations for each feature (plus extra scenarios when options are mutually exclusive).
6. Run terraform fmt -recursive before finishing.
7. Do not run terraform validate for this repository (module + provider alias limitation).

Expected Station test structure

Use this shape unless a feature needs a justified variation:

1. Provider declarations:
   • provider "tfe" {}
   • provider "azuread" {}
   • provider "azurerm" { features {} }
   • provider "azurerm" { alias = "connectivity" ... }
2. test { parallel = true }
3. Bootstrap/setup run blocks
4. File-level variables with placeholders such as "# Overridden" or "This has to be overridden"
5. Main run block(s) with:
   • module { source = "./" }
   • variables overrides using merge(...) and values from setup runs
   • assert blocks with clear error_message

Setup modules and when to use them

• tests/setup-common:
  • common tenant/subscription/identity data
  • frequently used for Graph/service principal data and current caller context
• tests/setup-tfe-project:
  • creates a dedicated TFC project for the test
  • output id is used to override var.tfe.project.id
• tests/setup-application:
  • bootstraps resource object IDs used by required_resource_access
• tests/setup-groups:
  • creates user/group fixtures for group membership scenarios
• tests/setup-entraid:
  • creates Entra ID fixtures for identity-related tests
• tests/setup-peering-networks:
  • creates remote VNet used by connectivity peering tests
• tests/setup-uai:
  • creates fixtures for UAI role/group assignment scenarios

Canonical override patterns

Use these exact ideas when placeholders exist in file-level variables.

Override TFC project ID

In main runs:

tfe = merge(var.tfe, { project = merge(var.tfe.project, { id = run.bootstrap_create_tfc_test_project.id }) })

Override setup-generated object IDs

For app tests and similar cases, map outputs from setup runs into nested feature objects with merge(...).

Examples:

• owners from current identity object ID
• required_resource_access.*.resource_object_id from setup run outputs
• connectivity peering remote_virtual_network_id from setup network output

How to add or update a Station feature test

1. Identify affected feature inputs and outputs in root module files.
2. Reuse an existing feature test file if one exists.
3. Ensure setup runs exist for all dynamic dependencies.
4. Add/adjust file-level variables to include:
   • minimal scenario
   • maximum scenario
   • extra scenario(s) only when needed
5. In main run variables, override placeholders from setup outputs.
6. Add assertions for:
   • input-to-resource/output mapping
   • default behavior
   • optional behavior in maximum scenario
   • required identity/app role/directory role behavior where relevant
7. Keep assertion messages specific and actionable.
8. Format and run targeted tests.

Assertions guidance (Station-specific)

• Prefer deterministic checks over brittle string comparisons.
• Use alltrue([...]) when validating repeated patterns across maps.
• For sensitive TFE vars, assert unreadable value behavior (value == "") and sensitive == true.
• Where appropriate, assert role assignments required by configured feature blocks (for example when var.applications or var.groups is set).
• If comparing structured HCL-like serialized output, normalize/parse before assertion (as done in existing tests/tfe.tftest.hcl).

Commands

Local prerequisites

• Azure CLI authenticated to correct tenant/subscription
• Terraform CLI installed
• Terraform Cloud credentials available
• Environment variables provided through a repo-root .agent.test.env file (copy from .agent.test.env.example)

Environment variable policy (required)

To avoid polluting shell state, never rely on globally exported variables for test runs.

Use this policy:

1. Keep all required variables in ./.agent.test.env (repo root).
2. For every Terraform command, source .agent.test.env inside a short-lived shell.
3. Do not run export ... directly in the interactive terminal session.

Agent workflow when asked to run tests:

1. Check whether ./.agent.test.env exists.
2. If missing, create it from ./.agent.test.env.example.
3. Stop and ask the user to fill real values in ./.agent.test.env before continuing.
4. Run Azure context preflight check (.agents/skills/terraform-station-test/scripts/check-az-context.sh) before Terraform commands.
5. Run Terraform commands only via the short-lived shell pattern below.

Existence/bootstrap command:

if [ ! -f ./.agent.test.env ]; then cp ./.agent.test.env.example ./.agent.test.env; echo "Created .agent.test.env from template. Fill values and re-run."; exit 1; fi

Azure context preflight command:

sh ./.agents/skills/terraform-station-test/scripts/check-az-context.sh ./.agent.test.env

This check ensures all of the following before tests proceed:

• az is installed
• user is logged in
• logged-in tenant matches TF_VAR_tenant_id
• active subscription matches TF_VAR_subscription_id
• TF_VAR_subscription_id equals ARM_SUBSCRIPTION_ID
• TFE_TOKEN is set for Terraform Cloud operations (including teardown)

Important: this does not force re-login each test. It only validates current context and fails fast when mismatched.

Preferred command pattern:

sh -c '. ./.agent.test.env; terraform <command>'

This keeps the environment scoped to that command only.

Typical workflow

sh ./.agents/skills/terraform-station-test/scripts/check-az-context.sh ./.agent.test.env
sh -c '. ./.agent.test.env; terraform init'
sh -c '. ./.agent.test.env; terraform fmt -recursive'
sh -c '. ./.agent.test.env; terraform test -filter=tests/<feature>.tftest.hcl'

Run all tests only when needed:

sh -c '. ./.agent.test.env; terraform test'

Quick preflight check:

sh -c '. ./.agent.test.env; env | grep -E "^(ARM_SUBSCRIPTION_ID|TF_VAR_subscription_id|TF_VAR_tenant_id|TFE_ORGANIZATION|TF_VAR_tfc_organization_name|TF_VAR_tfc_project_name)="'

# Optional explicit check for TFE token presence
sh -c '. ./.agent.test.env; test -n "${TFE_TOKEN:-}" && echo "TFE_TOKEN set" || (echo "TFE_TOKEN missing"; exit 1)'

CI behavior you must respect

CI uses selective test execution (.github/scripts/determine-tests.sh).

Implications:

• If you change only one feature area, prefer updating only that feature's test file.
• If you change core files or setup modules, expect all tests to run.
• If needed on PRs, maintainers can trigger all tests with /test-all or /test all.

Reference mapping docs:

• .github/scripts/README.md
• .github/workflows/terraform.yaml

Definition of done for test changes

• Test file follows Station structure and style used in existing tests/*.tftest.hcl
• Setup runs are present and outputs are used for dynamic overrides
• Minimum + maximum scenarios are covered (plus extra scenarios when justified)
• Assertions verify key behavior and required side effects
• terraform fmt -recursive has been run
• Relevant terraform test -filter=... command has been run (or command provided if execution is not possible)

When to ask for clarification

Ask before implementation only if one of these is unclear:

• Which feature/test file should be updated
• Whether new behavior is minimum, maximum, or separate scenario
• Whether setup fixture changes are acceptable
• Whether CI-impacting broad changes are intended

Copy/paste test skeletons

Use these as starting points and adjust names/variables/assertions to your feature.

Skeleton 1: New Station feature test file

provider "tfe" {}

provider "azuread" {}

provider "azurerm" {
	features {}
}

provider "azurerm" {
	alias = "connectivity"
	features {}
}

test {
	parallel = true
}

run "bootstrap_create_tfc_test_project" {
	variables {
		tfc_project_name = "tests_<feature>"
	}

	module {
		source = "./tests/setup-tfe-project"
	}
}

run "setup" {
	module {
		source = "./tests/setup-common"
	}
}

variables {
	tfe = {
		project = {
			id   = "# Overridden"
			name = "tests_<feature>"
		}
		organization_name     = "<org>"
		workspace_name        = "<feature>-test"
		workspace_description = "Station test for <feature>"
		workspace_settings = {
			execution_mode = "remote"
		}
	}

	<feature> = {
		minimum = {
			# required fields only
		}
		maximum = {
			# required + optional fields
		}
	}
}

run "<feature>-main" {
	variables {
		tfe = merge(var.tfe, {
			project = merge(var.tfe.project, {
				id = run.bootstrap_create_tfc_test_project.id
			})
		})
	}

	module {
		source = "./"
	}

	assert {
		condition     = true
		error_message = "Replace with real assertion for minimum scenario"
	}

	assert {
		condition     = true
		error_message = "Replace with real assertion for maximum scenario"
	}
}

Skeleton 2: Dynamic override pattern from setup outputs

Use this pattern when file-level values must be replaced by IDs from setup runs.

run "<feature>-main" {
	variables {
		<feature> = merge(var.<feature>, {
			maximum = merge(var.<feature>.maximum, {
				owners = [run.setup.azuread_client_config.current.object_id]

				required_resource_access = merge(var.<feature>.maximum.required_resource_access, {
					graph = merge(var.<feature>.maximum.required_resource_access.graph, {
						resource_object_id = run.bootstrap_application.MicrosoftGraph.object_id
					})
				})
			})
		})

		tfe = merge(var.tfe, {
			project = merge(var.tfe.project, {
				id = run.bootstrap_create_tfc_test_project.id
			})
		})
	}

	module {
		source = "./"
	}
}

Skeleton 3: Expected failure validation

Use when you intentionally verify input validation behavior.

run "<feature>-invalid-input" {
	command = plan

	variables {
		<feature> = {
			minimum = {
				# intentionally invalid value
			}
		}
	}

	module {
		source = "./"
	}

	expect_failures = [
		var.<feature>
	]
}

After adapting a skeleton:

• run sh -c '. ./.agent.test.env; terraform fmt -recursive'
• run sh -c '. ./.agent.test.env; terraform test -filter=tests/<feature>.tftest.hcl'
• add tighter assertions before opening a PR