Ron and Ella Wiki Page

Extremely Serious

Understanding State with show, state, and output

Terraform’s state is how it “remembers” what exists in your infrastructure so it can plan precise, minimal changes instead of blindly recreating resources. In this article, we’ll treat Terraform as a black box and learn how to inspect its memory using three key CLI tools: terraform show, the terraform state subcommands, and terraform output.


1. What Terraform State Actually Is

Terraform keeps a mapping between:

  • Your configuration (.tf files)
  • The real resources in your cloud provider (IDs, IPs, ARNs, etc.)

This mapping lives in a state file, usually terraform.tfstate, and often in a remote backend such as S3, Azure Blob Storage, or GCS for team use. The state includes every attribute of every managed resource, plus metadata used for things like dependency ordering and change detection.

Why you care:

  • Debugging: Is Terraform seeing the same thing you see in the console?
  • Refactoring: How do you rename resources without destroying them?
  • Automation: How do you feed outputs into CI/CD or other tools?

You should never hand-edit the state file; instead you use the CLI commands discussed below to read or safely modify it.


2. terraform show — Inspecting the Whole State or a Plan

Think of terraform show as “dump what Terraform currently knows” — it turns a state file or a saved plan into a human-readable or JSON view.

Core usage

# Show the current state snapshot (from the active backend)
terraform show

# Show a specific state file
terraform show path/to/terraform.tfstate

# Show a saved plan file
terraform show tfplan

# Machine-readable JSON for tooling
terraform show -json > plan.json
  • Without a file argument, terraform show prints the latest state snapshot from the active backend.
  • With a plan file, it describes the proposed actions and resulting state.
  • With -json, you get a structured document that external tools (e.g. CI, tests) can parse and validate.

Important: When using -json, sensitive values are printed in plain text; handle this carefully in pipelines and logs.

When to use terraform show

Use it when:

  • You want a global view: “What exactly is Terraform tracking right now?”
  • You want to inspect a plan artifact (plan -out tfplan) before approving it in CI.
  • You want to feed state or plan data into a tool (via -json) for policy checks, drift checks, or custom validation.

Conceptually, terraform show is read-only and holistic: it treats the state (or plan) as a whole, rather than individual resources.


3. terraform state — Fine-Grained State Inspection and Surgery

The terraform state command is a group of subcommands designed specifically to inspect and modify state without touching real infrastructure. This is the surgical toolkit you reach for when refactoring or repairing.

Key subcommands

Command What it does Typical use
terraform state list Lists all resource addresses in state “What is Terraform tracking?”
terraform state show ADDRESS Shows attributes of one resource Debugging one resource (IDs, IPs, tags, etc.)
terraform state mv SRC DEST Moves/renames a resource in state Refactors config without destroy/recreate
terraform state rm ADDRESS Removes a resource from state Stop managing a resource without deleting it
terraform state pull Prints raw state to stdout Backup, inspection, or external processing
terraform state push Uploads a local state file Restore/correct broken remote state (used rarely, carefully)

3.1 terraform state list

terraform state list
# e.g.
# aws_instance.web[0]
# aws_instance.web[1]
# aws_security_group.allow_ssh

This gives you the resource addresses Terraform knows about, optionally filtered by a prefix. It’s extremely useful when working with modules or count/for_each, because you can see the exact address Terraform expects.

3.2 terraform state show

terraform state show aws_instance.web[0]

This prints every attribute of that specific resource as seen in state — IDs, IPs, tags, relationships, and computed attributes. Semantically, it answers: “What does Terraform think this one resource looks like?”.

Use it when:

  • Debugging drift: console vs state mismatch.
  • Understanding complex resources: which subnet, which IAM role?
  • Checking data sources that were resolved at apply time.

Note the difference:

  • terraform show → everything (or full plan).
  • terraform state show ADDRESS → one resource only.

3.3 terraform state mv — Refactor Without Downtime

terraform state mv aws_instance.web aws_instance.app

If you simply rename the block in your .tf code, Terraform will plan to destroy the old resource and create a new one because it assumes they’re unrelated. state mv tells Terraform that the underlying resource is the same, you’re just changing the mapping.

This is critical for:

  • Renaming resources.
  • Moving resources into/out of modules.
  • Splitting a monolith configuration into multiple modules/workspaces.

3.4 terraform state rm — Stop Managing Without Deleting

terraform state rm aws_instance.legacy

This removes the resource from Terraform’s management while leaving it alive in your provider. Use this when decommissioning Terraform from part of your estate or when you temporarily need Terraform to “forget” something (e.g. migration to a different tool).

3.5 terraform state pull / push

These expose and manipulate the raw state blob:

terraform state pull > backup.tfstate
terraform state push backup.tfstate

They’re useful for backups or extremely rare recovery scenarios, but they’re dangerous if misused, so in practice you rely much more on list, show, mv, and rm.


4. terraform output — Consuming State Safely

terraform output reads output values defined in the root module and prints their values from the state file. It is the “official interface” for other systems (and humans) to consume selected bits of state without parsing the state file directly.

4.1 Defining outputs in configuration

In your root module:

output "instance_ips" {
  value = aws_instance.web[*].public_ip
}

output "lb_address" {
  value = aws_lb.web.dns_name
}

output "db_connection_string" {
  value     = module.database.connection_string
  sensitive = true
}
  • Outputs are calculated after terraform apply and stored in state.
  • Only root module outputs are visible to terraform output; child module outputs must be re-exposed.

4.2 Using terraform output interactively

# Show all outputs for the root module
terraform output

# Show one specific output
terraform output lb_address

# Machine-readable JSON
terraform output -json

# Raw string (no quotes/newlines), perfect for scripts
terraform output -raw lb_address
  • With no arguments, it prints all root outputs.
  • With a NAME, it prints just that value.
  • -json gives a JSON object keyed by output name; can be piped into jq or similar tools.
  • -raw prints a bare string/number/boolean; ideal when exporting in shell scripts without extra quoting.

This is the idiomatic way to feed state into:

  • CI/CD pipelines (e.g. get ALB DNS for integration tests).
  • Other scripts (e.g. configure DNS records).
  • Other tools (e.g. Ansible inventory).

5. Putting It Together: A Simple Example

Below is a minimal, self-contained configuration you can run locally.

5.1. Prerequisites

  1. LocalStack running (Docker is typical):

    docker run --rm -it -p 4566:4566 -p 4510-4559:4510-4559 localstack/localstack

    LocalStack’s edge endpoint is exposed on http://localhost:4566 by default.

  2. Terraform installed (1.x).

5.2. Terraform Configuration Using LocalStack

Create a directory (for example tf-localstack-ec2) and within it create two files: versions.tf and main.tf.

versions.tf

Lock AWS provider to a version that is known to work well with LocalStack (4.x is a common choice):

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

main.tf

provider "aws" {
  region                      = "us-east-1"
  access_key                  = "test"
  secret_key                  = "test"
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true

  endpoints {
    ec2 = "http://localhost:4566"
  }
}

resource "aws_instance" "web" {
  ami           = "ami-12345678"
  instance_type = "t3.micro"

  tags = {
    Name = "tf-demo-web-localstack"
  }
}

output "web_public_ip" {
  value = aws_instance.web.public_ip
}

Notes:

  • The endpoints.ec2 block points the EC2 API at LocalStack’s edge endpoint.
  • Credentials are dummy; LocalStack doesn’t actually validate them.
  • The AMI ID is a placeholder; LocalStack typically does not require the AMI to exist, but EC2 support is limited and can hang for some combinations. For state/command learning it’s usually enough that Terraform “thinks” it created something.

5.3. How to Apply and Validate

From the directory containing these files:

  1. Initialize and apply

    terraform init
    terraform apply -auto-approve

    Terraform will talk to LocalStack instead of AWS because of the custom endpoint.

  2. Validate with show

    terraform show

    Confirm there is a aws_instance.web block with attributes populated from LocalStack’s response.

  3. Validate with state

    terraform state list
    # should include:
    # aws_instance.web
    
    terraform state show aws_instance.web

    This tells you what Terraform’s state holds for this specific resource address.

  4. Validate with output

    terraform output web_public_ip
    terraform output -raw web_public_ip

    For LocalStack, the public IP may be empty or synthetic depending on EC2 emulation level, but the command proves the wiring from resource → state → output.

5.4. Rationale for These Choices

  • We override only the EC2 endpoint to keep the example close to “real” AWS code while still talking to LocalStack.
  • We relax provider validations (skip_* flags) because LocalStack does not implement all AWS account/metadata APIs.

With this setup, you can safely experiment with terraform show, terraform state *, and terraform output on your laptop, without touching real AWS accounts or incurring cost.


6. Conceptual Summary: Which Command When?

Need Command Rationale
See everything Terraform knows terraform show Whole-state, read-only view (or a plan)
Inspect one resource deeply terraform state show ADDRESS Focused, per-resource state inspection
List all tracked resources terraform state list Discover resource addresses in state
Rename/move resources/modules terraform state mv Refactor mappings without downtime
Forget a resource but keep it alive terraform state rm Stop managing without deleting
Give other tools a clean interface terraform output / -json / -raw Official way to expose selected state data developer.

The underlying rationale is separation of concerns:

  • terraform showobservability of plans and state.
  • terraform stateprecise manipulation and inspection of state.
  • terraform outputcontrolled, stable API to state for humans and downstream systems.

Terraform Console in Practice: Your Interactive HCL Lab

Terraform console is an interactive interpreter where you can evaluate Terraform expressions, inspect state, and prototype logic before committing anything to code or infrastructure.


1. What terraform console actually is

At its core, terraform console is a REPL for Terraform’s expression language (HCL2).

  • It reads your configuration and current state from the configured backend, so you can query real values: var.*, local.*, resource.*, data.*.
  • It is read‑only with respect to infrastructure: it does not change resources or configuration, it only evaluates expressions against configuration/state or, if you have no state yet, against pure expressions and built‑ins.
  • It holds a lock on state while open, so other commands that need the state (plan/apply) will wait or fail until you exit.

Pedagogically, think of it as Terraform’s “maths lab”: you experiment with expressions and data structures in isolation before wiring them into modules.


2. Why you should care as a practitioner

You will use terraform console for three broad reasons:

  • Rapid feedback on expressions
    • Test for expressions, conditionals, complex locals, and functions like cidr*, jsonencode, jsondecode, file, etc., without running full plans.
  • Insight into “what Terraform thinks”
    • Inspect live values for resources, data sources, variables, and outputs as Terraform sees them in state, which is often where misunderstandings hide.
  • Debugging complex data structures
    • When for_each over nested maps/lists behaves oddly, you can print and transform the structures interactively to understand shape and keys before editing code.

This shortens the debug loop significantly on large stacks and reduces the risk of generating enormous, accidental plans.


3. Running the console and basic usage

In any initialized working directory:

terraform init    # if not already done
terraform console

You then get a prompt like:

> 1 + 2
3

> upper("auckland")
"AUCKLAND"

You can reference configuration components directly:

> var.cidr
"10.0.0.0/24"

> cidrnetmask(var.cidr)
"255.255.255.0"

> cidrhost(var.cidr, 10)
"10.0.0.10"

Inspecting resources and data sources:

> aws_s3_bucket.data
# prints the entire state object of that bucket (attributes, tags, region, etc.)

Terraform’s own tutorial demonstrates this pattern with an S3 bucket, using terraform console to print attributes like bucket, arn, region, ACLs and so on from state.

To exit:

> exit

Or press Ctrl+D / Ctrl+C.


4. Evaluating expressions: from simple to advanced

The console supports essentially any expression you can write in HCL: literals, operators, functions, for expressions, conditionals, etc.

Examples:

  • Lists and maps:

    > [for env in ["dev", "test", "prod"] : "env-${env}"]
    [
    "env-dev",
    "env-test",
    "env-prod",
    ]
    
    > { for k, v in { a = 1, b = 2, c = 3 } : k => v if v % 2 == 1 }
    {
    "a" = 1
    "c" = 3
    }
  • Filtering complex maps (example adapted from the docs):

    variable "apps" {
    type = map(any)
    default = {
      foo = { region = "us-east-1" }
      bar = { region = "eu-west-1" }
      baz = { region = "ap-south-1" }
    }
    }

    In the console:

    > var.apps.foo
    {
    "region" = "us-east-1"
    }
    
    > { for key, value in var.apps : key => value if value.region == "us-east-1" }
    {
    "foo" = {
      "region" = "us-east-1"
    }
    }
  • Testing network helpers:

    > cidrnetmask("172.16.0.0/12")
    "255.240.0.0"
    ```[1]

This is exactly how you should design locals and for_each expressions: prototype an expression in console, inspect the result, then paste into your module.


5. Inspecting state and outputs

Console is wired to your current backend and workspace.

  • Inspect an entire resource instance:

    > aws_s3_bucket.data
    # large object showing bucket name, ARN, tags, region, ACL, encryption, etc.

    The S3 tutorial shows this in detail, where the console prints attributes like bucket, bucket_domain_name, force_destroy, encryption configuration, tags, and more.

  • Build structured objects and validate them:

    > jsonencode({
      arn    = aws_s3_bucket.data.arn
      id     = aws_s3_bucket.data.id
      region = aws_s3_bucket.data.region
    })
    "\"{\\\"arn\\\":\\\"arn:aws:s3:::...\\\",\\\"id\\\":\\\"...\\\",\\\"region\\\":\\\"us-west-2\\\"}\""

The tutorial uses this pattern to design an output bucket_details, then later validates that terraform output -json bucket_details produces the exact desired JSON structure.

This is a powerful workflow: design your JSON structures interactively in console, then turn them into outputs or policy documents.


6. Using the console with plans (-plan)

By default, console evaluates expressions against the current state, which means values “known after apply” are not concrete yet.

You can ask console to evaluate against a fresh plan:

terraform console -plan

Now you can inspect “planned” values that do not exist in state yet, e.g. resources that are about to be created.

Rationale: this helps reason about the result of for_each, count, and complex expressions before touching real infrastructure. The docs do note that configurations which perform side effects during planning (for example via external data sources) will also do so in console -plan, so such patterns are discouraged.


7. Non‑interactive/scripting usage

You can pipe expressions into console from a script; only the last expression’s result is printed unless an error occurs.

Example from the reference:

echo 'split(",", "foo,bar,baz")' | terraform console

Output:

tolist([
  "foo",
  "bar",
  "baz",
])

This is extremely handy for:

  • CI checks that assert a particular expression evaluates to an expected structure.
  • One‑off debugging scripts that compute derived values from state (e.g. join tags, summarise regions) without adding permanent outputs.

8. A simple example

Let’s assemble a minimal, end‑to‑end example that you can run locally.

8.1. Configuration

Files:

variables.tf:

variable "cidr" {
  type    = string
  default = "10.0.0.0/24"
}

main.tf:

terraform {
  required_version = ">= 1.1.0"

  required_providers {
    random = {
      source  = "hashicorp/random"
      version = "~> 3.0"
    }
  }
}

provider "random" {}

resource "random_password" "db" {
  length  = 16
  special = true
}

locals {
  subnet_ips = [
    for host in range(1, 5) :
    cidrhost(var.cidr, host)
  ]
}

output "db_password" {
  value     = random_password.db.result
  sensitive = true
}

output "subnet_ips" {
  value = local.subnet_ips
}

This uses standard providers and functions: random_password, cidrhost, range, and a for expression, all supported in Terraform 1.1+.

8.2. Apply once

terraform init
terraform apply -auto-approve

You now have state with random_password.db and all locals resolved.

8.3. Explore and validate with terraform console

Run:

terraform console

Try these expressions:

> var.cidr
"10.0.0.0/24"

> local.subnet_ips
[
  "10.0.0.1",
  "10.0.0.2",
  "10.0.0.3",
  "10.0.0.4",
]

> random_password.db.result
"R@nd0mP@ss..." # your value will be different

Validation steps:

  1. Confirm outputs match console:

    terraform output subnet_ips

    You should see the same list printed that you saw for local.subnet_ips in console. Both are derived from the same expression and state.

  2. Confirm password consistency:

    • terraform state show random_password.db will show the result field.
    • Compare that value with random_password.db.result printed in console; they must be identical for the same state snapshot.

If both checks pass, you have empirically validated that:

  • The console is looking at the same state as terraform state and terraform output.
  • Your locals and for expressions behave exactly as expected before you embed similar patterns into more complex modules.

9. Rationale and best‑practice use

From an engineering‑practice perspective, use terraform console as a standard part of your workflow:

  • Before adding non‑trivial expressions
    • Prototype them in console with realistic variable values; only once you’re happy paste them into locals or resource arguments.
  • When debugging bugs in production stacks
    • Inspect what Terraform actually has in state for a resource or data source, rather than inferring from code.

Used this way, console is not a “nice extra” but a core tool: it turns Terraform’s somewhat opaque expression runtime into something you can interrogate directly and safely.

Terraform Expressions and Functions

Terraform’s expression and function system is the core “thinking engine” behind your configurations: expressions describe what value an argument should have, and functions are reusable tools you invoke inside those expressions to compute values dynamically.


1. What Is an Expression in Terraform?

An expression is any piece of HCL that Terraform can evaluate to a concrete value: a string, number, bool, list, map, or object. You use expressions in almost every place on the right-hand side of arguments, in locals, count, for_each, dynamic blocks, and more.

Common expression forms:

  • Literals: "hello", 5, true, null, ["a", "b"], { env = "dev" }.
  • References: var.region, local.tags, aws_instance.app.id, module.vpc.vpc_id.
  • Operators: arithmetic (+ - * / %), comparisons (< > <= >=), equality (== !=), logical (&& || !).
  • Conditionals: condition ? value_if_true : value_if_false.
  • for expressions: [for s in var.subnets : upper(s)] to transform collections.
  • Splat expressions: aws_instance.app[*].id to project attributes out of collections.

Rationale: Terraform must stay declarative (you describe the desired state), but real infrastructure is not static; expressions give you a minimal “language” to derive values from other values without dropping into a general-purpose programming language.


2. What Is a Function?

A function is a built‑in helper you call inside expressions to transform or combine values. The syntax is a function name followed by comma‑separated arguments in parentheses, for example max(5, 12, 9). Functions always return a value, so they can appear anywhere a normal expression is allowed.

Key properties:

  • Terraform ships many built‑in functions (string, numeric, collection, IP/network, crypto, time, type conversion, etc.).
  • You cannot define your own functions in HCL; you only use built‑ins, plus any provider-defined functions a provider may export.
  • Provider-defined functions are namespaced like provider::<local-name>::function_name(...) when used.

Examples of useful built‑in functions:

  • String: upper("dev"), lower(), format(), join("-", ["app", "dev"]).
  • Numeric: max(5, 12, 9), min(), ceil(), floor().
  • Collection: length(var.subnets), merge(local.tags, local.extra_tags), flatten().

Rationale: Functions cover the common transformation needs (naming, list/map manipulation, math) so that your Terraform remains expressive but compact, and you avoid copy‑pasting “string‑mangling” logic everywhere.


3. How Expressions and Functions Work Together

Terraform’s model is expression‑centric: on the right‑hand side of almost every argument, you write an expression, and function calls are just one kind of expression. You freely compose references, operators, conditionals, for expressions, and functions, as long as the input and output types match.

Typical composition patterns:

  • Use references (var.*, local.*, resource attributes) as the base inputs.
  • Apply operators and conditional expressions to make decisions (var.env == "prod" ? 3 : 1).
  • Use for expressions and collection functions to reshape data ([for s in var.subnets : upper(s)]).
  • Use string functions to build consistent resource names (format("%s-%s", var.app, var.env)).

From a mental-model perspective, a good way to think about this is: “Everything dynamic lives in expressions; functions are building blocks inside those expressions.”


4. A Small Example

Below is a minimal Terraform configuration that showcases expressions and functions together, and that you can actually run to observe evaluation results.

Example configuration (main.tf)

terraform {
  required_version = ">= 1.6"
}

variable "environment" {
  type    = string
  default = "dev"
}

variable "app_servers" {
  type    = list(string)
  default = ["app-1", "app-2", "app-3"]
}

locals {
  # Expression: equality operator -> bool
  is_prod = var.environment == "prod"

  # Literal map and reference
  base_tags = {
    app         = "payments"
    environment = var.environment
  }

  # For expression + string function
  uppercased_servers = [for s in var.app_servers : upper(s)]

  # Merge and format functions to compute a name once
  common_tags = merge(
    local.base_tags,
    {
      name = format(
        "%s-%s-%02d",
        local.base_tags.app,
        local.base_tags.environment,
        length(var.app_servers)
      )
    }
  )
}

output "summary" {
  value = {
    is_prod            = local.is_prod
    uppercased_servers = local.uppercased_servers
    common_tags        = local.common_tags
  }
}

What this demonstrates conceptually:

  • Expressions:
    • var.environment == "prod" produces a bool for local.is_prod.
    • The map in local.base_tags uses both literals and references.
    • The locals block itself is a way to give names to intermediate expressions.
  • Functions:
    • upper(s) transforms each server name to uppercase inside a for expression.
    • length(var.app_servers) computes the number of servers.
    • format("%s-%s-%02d", ...) builds a stable name string.
    • merge(...) combines two maps into a single tag map.

Rationale: This pattern—variables + locals + expressions + functions—is exactly how you avoid repetition and keep a production Terraform codebase readable as it grows.


5. How to Validate This Example

Terraform provides an expression console and standard workflow commands to validate that your expressions and functions behave as expected before they affect real infrastructure.

Option A: Run the configuration

  1. Save the example as main.tf in an empty directory.
  2. Run:
    • terraform init to set up the working directory.
    • terraform apply -auto-approve to evaluate and show outputs.
  3. Observe the summary output:
    • is_prod should be false (with the default environment dev).
    • uppercased_servers should be ["APP-1", "APP-2", "APP-3"].
    • common_tags.name should be payments-dev-03.

To see how expressions react to different inputs, run again with a different environment:

terraform apply -auto-approve -var 'environment=prod'

Now is_prod will be true, and the computed name will switch to payments-prod-03, even though you haven’t changed any resource definitions.

Option B: Experiment interactively with terraform console

Terraform’s console lets you test expressions and functions on the fly.

From the same directory:

terraform console

Then try:

> 1 + 2 * 3
> var.environment == "prod"
> [for s in var.app_servers : upper(s)]
> merge({a = 1}, {b = 2})
> format("%s-%s", "app", var.environment)

You will see the evaluated results immediately, which is ideal for teaching yourself how a particular expression or function behaves before embedding it into a real module.


6. Summary Table: Expressions vs Functions

Aspect Expressions Functions
Purpose Describe how to compute a value. Provide reusable operations used inside expressions.
Examples var.env == "prod", [for s in xs : x.id]. length(var.subnets), join("-", local.tags).
Defined by Terraform language syntax. Terraform’s built‑in and provider-defined function library.
Customization Composed via locals, variables, and blocks. No user-defined functions in HCL; only built‑ins/providers.
Typical usage domain Conditionals, loops, references, constructing structures. String formatting, math, collection manipulation, conversion.

Terraform Format and Validation

In Terraform projects, format and validation are your first line of defence against messy code and avoidable runtime errors. Think of them as “style checking” and “sanity checking” for infrastructure as code.


Why Format and Validate at All?

Terraform configurations tend to grow into large, multi‑module codebases, often edited by several engineers at once. Without conventions and guards:

  • Small style differences accumulate into noisy diffs.
  • Subtle typos, type mismatches, or broken references sneak into main.
  • Misused modules cause surprises late in plan or even apply.

Formatting (terraform fmt) standardises how the code looks, while validation (terraform validate and variable validation) standardises what values are acceptable.


Formatting Terraform Code with terraform fmt

What terraform fmt Actually Does

terraform fmt rewrites your .tf files into Terraform’s canonical style.

It:

  • Normalises indentation and alignment.
  • Orders and spaces arguments consistently.
  • Applies a single canonical style across the project.

Typical usage:

# Fix formatting in the current directory
terraform fmt

# Recurse through modules and subfolders (what you want in a real repo)
terraform fmt -recursive

For CI or pre‑commit hooks you almost always want:

terraform fmt -check -recursive

This checks formatting, returns a non‑zero exit code if anything is off, but does not modify files. That makes it safe for pipelines.

Why This Matters Architecturally

  • Consistent formatting reduces cognitive load; you can scan resources quickly instead of re‑parsing everyone’s personal style.
  • Diffs stay focused on behaviour instead of whitespace and alignment.
  • A shared style is essential when modules are reused across teams and repos.

Treat terraform fmt like go fmt: it’s not a suggestion, it’s part of the toolchain.


Structural Validation with terraform validate

What terraform validate Checks

terraform validate performs a static analysis of your configuration for syntactic and internal consistency.

It verifies that:

  • HCL syntax is valid.
  • References to variables, locals, modules, resources, and data sources exist.
  • Types are consistent (for example you’re not passing a map where a string is expected).
  • Required attributes exist on resources and data blocks.

Basic usage:

terraform init     # required once before validate
terraform validate

If everything is fine you will see:

Success! The configuration is valid.

This does not contact cloud providers; it is a “compile‑time” check, not an integration test.terraformpilot+1

Why You Want It in Your Workflow

  • Catches simple but common mistakes (typos in attribute names, missing variables, wrong types) before plan.
  • Cheap enough to run on every commit and pull request.
  • Combined with fmt, it gives you a fast gate that keeps obviously broken code out of main.

In CI, a very standard pattern is:

terraform fmt -check -recursive
terraform init -backend=false   # or with backend depending on your setup
terraform validate

You can choose whether init uses the real backend or a local one; the key is that validate runs automatically.


Input Variable Validation: Types and Rules

Terraform also validates values going into your modules via input variables. There are three important layers.

1. Type Constraints

Every variable can and should declare a type: string, number, bool, complex types such as list(string) or object({ ... }). Terraform will reject values that do not conform.

Example:

variable "tags" {
  type = map(string)
}

Passing a list here fails fast, long before any resource is created.

2. Required vs Optional

  • Variables without a default are required; if the caller does not supply a value, Terraform fails at validation time.
  • Variables with a default are optional; they still participate in type and custom validation.

This lets you express what callers must always provide versus what can be inferred or defaulted.

3. Custom validation Blocks

Inside each variable block you can define one or more validation blocks.

Each block has:

  • condition: a boolean expression evaluated against the value.
  • error_message: a human‑readable message if the condition is false.

Example patterns from common practice include:

  • Membership checks with contains or regex.
  • Ranges and integer checks for numbers.
  • Multiple validation blocks to capture several independent rules.

The rationale here is strong: you make invalid states unrepresentable at the module boundary, rather than having to handle them deep inside resource logic.


Beyond Variables: Preconditions and Postconditions

Terraform also lets you validate assumptions around resources and data sources using precondition and postcondition blocks.

  • A precondition asserts something must be true before Terraform creates or updates the object (for example, an input computed from multiple variables is within bounds).
  • A postcondition asserts something must be true after the resource or data source is applied (for example, an attribute returned by the provider matches expectations).

Conceptually:

  • Variable validation guards inputs to modules.
  • Preconditions/postconditions guard behaviour of resources and data sources exposed by those modules.

For a team consuming your module, this is powerful: they get immediate, clear errors about violated invariants instead of mysterious provider failures later.


A Simple Example (Format + Validate + Variable Rules)

Below is a small, self‑contained configuration you can run locally to see formatting and validation in action.

Files

Create the files.

variables.tf:

variable "environment" {
  description = "Deployment environment."
     type        = string

  validation {
    condition     = contains(["dev", "test", "prod"], var.environment)
    error_message = "Environment must be one of: dev, test, prod."
  }
}

variable "app_name" {
  description = "Short application name used in resource naming."
  type        = string

  validation {
    condition     = can(regex("^[a-z0-9-]{3,20}$", var.app_name))
    error_message = "app_name must be 3-20 chars, lowercase letters, digits, and hyphens only."
  }
}

variable "instance_count" {
  description = "Number of instances to run."
  type        = number

  validation {
    condition     = var.instance_count >= 1 && var.instance_count <= 10
    error_message = "instance_count must be between 1 and 10."
  }

  validation {
    condition     = !(var.environment == "prod" && var.instance_count < 3)
    error_message = "In prod, instance_count must be at least 3."
  }
}

This demonstrates:

  • Type constraints on all inputs.
  • A small “enumeration” for environment.
  • A format rule enforced via regex on app_name.
  • Multiple independent validation rules on instance_count, including one that depends on environment.

main.tf:

terraform {
  required_version = ">= 1.5.0"
}

locals {
  app_tag = "${var.app_name}-${var.environment}"
}

output "example_tags" {
  value = {
    Environment = var.environment
    App         = var.app_name
    Count       = var.instance_count
    AppTag      = local.app_tag
  }
}

Step 1 – Format the Code

From inside the folder:

terraform fmt -recursive

Observe that Terraform will adjust spacing/indentation if you intentionally misalign something and run it again. This confirms fmt is active and working.

Step 2 – Initialize

terraform init

No providers are actually used here, but validate requires initialization.

Step 3 – Structural Validation

Run:

terraform validate

This checks syntax, references, and type soundness of the configuration itself.

If you see:

Success! The configuration is valid.

you know the configuration is structurally sound.

Step 4 – Test Variable Validation with plan and -var

To exercise your variable validation logic with specific values, use terraform plan with -var flags.

  1. Valid input:

    terraform plan -var="environment=dev" -var="app_name=demo-app" -var="instance_count=2"
    • Here -var is supported and your custom validation blocks are evaluated.
    • This should succeed, producing a plan (no resources, but the important part is that there are no validation errors).
  2. Invalid environment:

    terraform plan -var="environment=stage" -var="app_name=demo-app" -var="instance_count=2"

    Expect Terraform to fail with the custom environment error message from the validation block.

  3. Invalid app name:

    terraform plan -var="environment=dev" -var="app_name=Demo_App" -var="instance_count=2"

    You should see the regex‑based app_name error.

  4. Invalid prod count:

    terraform plan -var="environment=prod" -var="app_name=demo-app" -var="instance_count=1"

    Here, the environment is valid and the type is correct, but the cross‑rule on instance_count fails with your custom prod message.

Optional – Use *.tfvars Instead of -var

If you prefer files over command‑line flags, create dev.auto.tfvars:

environment    = "dev"
app_name       = "demo-app"
instance_count = 2

Then just run:

terraform plan

Terraform will automatically load *.auto.tfvars files and apply the same variable validations.


Recommended Pattern for Teams

Updated to reflect current behaviour:

  • Run terraform fmt -check -recursive and terraform validate in CI on every PR.
  • Use terraform plan (with -var or *.tfvars) to exercise and gate variable validations for concrete environments (dev, test, prod).
  • Enforce types and validation blocks on all externally visible variables, not just a handful.
  • Use preconditions and postconditions where module consumers must rely on specific guarantees from your resources.

From an engineering‑lead perspective, this gives you a clear division of responsibilities:

  • fmt → canonical style.
  • validate → structural soundness of the configuration.
  • plan (with variables) → semantic correctness of inputs and module contracts.

Terraform Block Types

Terraform configurations are built out of blocks. Understanding block types is critical because they define how you declare infrastructure, wire modules together, and control Terraform’s behavior.


1. The Anatomy of a Block

Every Terraform block has the same basic shape:

TYPE "label1" "label2" {
  argument_name = expression

  nested_block_type {
    # ...
  }
}

Key parts:

  • Type: The keyword at the start (resource, provider, variable, etc.). This tells Terraform what kind of thing you are defining.
  • Labels: Extra identifiers whose meaning depends on the block type.
    • Example: resource "aws_instance" "web"
    • Type: resource
    • Labels: "aws_instance" (resource type), "web" (local name)
  • Body: The { ... } section, which can contain:
    • Arguments: name = expression
    • Nested blocks: block_type { ... }

Rationale: The consistent shape makes the language predictable. Block type + labels define what the block is; the body defines how it behaves or is configured.


2. Core Top-Level Block Types

These blocks usually appear at the top level of your .tf files and together they define a module: its inputs, logic, and outputs.

2.1 terraform block

Configures Terraform itself:

  • Required providers and their versions.
  • Required Terraform version.
  • Backend configuration (usually via a nested backend block in terraform).

Example:

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

Rationale: Keeps tooling constraints explicit and version-pinned, so behavior is deterministic across environments and team members.


2.2 provider block

Configures how Terraform talks to an external API (AWS, Azure, GCP, Kubernetes, etc.):

provider "aws" {
  region = var.aws_region
}

Typical aspects:

  • Credentials and regions.
  • Aliases for multiple configurations (e.g., provider "aws" { alias = "eu" ... }).

Rationale: Providers are the “drivers” Terraform uses to translate configuration into real infrastructure; separating them lets you re-use the same module with different provider settings.


2.3 resource block

Declares infrastructure objects Terraform will create and manage.

resource "aws_s3_bucket" "this" {
  bucket = "${local.name}-bucket"
}

Structure:

  • Type label: the provider-specific resource type ("aws_s3_bucket").
  • Name label: a local identifier ("this", "web", "db", etc.).
  • Body: arguments and nested blocks that define the resource’s configuration.

Rationale: The resource block is the heart of Terraform; it expresses desired state. Every apply tries to reconcile actual infrastructure with what these blocks declare.


2.4 data block

Reads information about existing objects without creating anything.

data "aws_ami" "latest_amazon_linux" {
  most_recent = true

  filter {
    name   = "name"
    values = ["amazon-linux-2-*"]
  }

  owners = ["amazon"]
}

You reference it as data.aws_ami.latest_amazon_linux.id.

Rationale: Data sources decouple “lookup” from “creation”. You avoid hardcoding IDs/ARNs and can dynamically discover things like AMIs, VPC IDs, or roles.


2.5 variable block

Defines inputs to a module:

variable "aws_region" {
  type        = string
  description = "AWS region to deploy into"
  default     = "us-west-2"
}

Key fields:

  • type: basic or complex types (string, number, list, map, object, etc.).
  • default: makes a variable optional.
  • description: documentation for humans.

Rationale: Explicit inputs make modules reusable, testable, and self-documenting. They are your module’s API.


2.6 output block

Exposes values from a module:

output "bucket_name" {
  value       = aws_s3_bucket.this.bucket
  description = "Name of the S3 bucket created by this module."
}

Rationale: Outputs are your module’s return values, allowing composition: root modules can print values, and child modules can feed outputs into other modules or systems (e.g., CI/CD).


2.7 locals block

Defines computed values for use within a module:

locals {
  name_prefix = "demo"
  bucket_name = "${local.name_prefix}-bucket"
}

Notes:

  • You can have multiple locals blocks; Terraform merges them.
  • Access them via local.<name>.

Rationale: Locals centralize derived values and remove duplication. That keeps your configuration DRY and easier to refactor.


3. Nested Blocks vs Arguments

Within a block body you use two constructs:

  • Arguments: key = expression
    Example: bucket = "demo-bucket".

  • Nested blocks: block_type { ... }
    Example:

    resource "aws_instance" "web" {
    ami           = data.aws_ami.latest_amazon_linux.id
    instance_type = "t3.micro"
    
    network_interface {
      device_index = 0
      network_interface_id = aws_network_interface.web.id
    }
    }

Why have both?

  • Arguments are single values; they are the usual “settings”.
  • Nested blocks model structured, often repeatable configuration sections (e.g., ingress rules in security groups, network_interface, lifecycle, tag blocks in some providers).

Rationale: Using nested blocks for structured/repeated sections keeps complex resources readable and makes it clear which values logically belong together.


4. Meta-Arguments and Lifecycle Blocks

Some names inside a resource are meta-arguments understood by Terraform itself rather than by the provider:

Common meta-arguments:

  • depends_on: Add explicit dependencies when Terraform’s graph inference isn’t enough.
  • count: Create multiple instances of a resource using integer indexing.
  • for_each: Create multiple instances keyed by a map or set.
  • provider: Pin a resource to a specific provider configuration (e.g., aws.eu).
  • lifecycle: Special nested block that controls create/update/destroy behavior.

Example lifecycle:

resource "aws_s3_bucket" "this" {
  bucket = "${local.name}-bucket"

  lifecycle {
    prevent_destroy       = true
    ignore_changes        = [tags]
    create_before_destroy = true
  }
}

Rationale: Meta-arguments give you control over resource orchestration rather than definition. They let you express cardinality, ordering, and safety rules without resorting to hacks or external tooling.


5. Putting It All Together

Below is a small but coherent configuration that demonstrates the main block types and how they interact. You can drop this into an empty directory as main.tf.

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

variable "aws_region" {
  type        = string
  description = "AWS region to deploy into (used by LocalStack as well)"
  default     = "us-east-1"
}

locals {
  project = "block-types-localstack-demo"
  bucket  = "${local.project}-bucket"
}

provider "aws" {
  region  = var.aws_region

  # Dummy credentials – LocalStack doesn’t actually validate them.
  access_key = "test"
  secret_key = "test"

  # Talk to LocalStack instead of AWS.
  s3_use_path_style           = true
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true

  endpoints {
    s3 = "http://localhost:4566"
    sts = "http://localhost:4566"
  }
}

# This data source will return the LocalStack “test” account (000000000000).
data "aws_caller_identity" "current" {}

resource "aws_s3_bucket" "this" {
  bucket = local.bucket

  tags = {
    Project = local.project
    Owner   = data.aws_caller_identity.current.account_id
  }

  lifecycle {
    prevent_destroy = true
  }
}

output "bucket_name" {
  value       = aws_s3_bucket.this.bucket
  description = "The name of the created S3 bucket."
}

output "account_id" {
  value       = data.aws_caller_identity.current.account_id
  description = "AWS (LocalStack) account ID used for this deployment."
}

What this example shows

  • terraform block: pins Terraform and the AWS provider.
  • variable: input for region.
  • locals: internal naming logic.
  • provider: AWS configuration.
  • data: a data source reading your current AWS identity.
  • resource: S3 bucket, including nested lifecycle and tags.
  • output: exposes bucket name and account ID.

How to Run and Validate with LocalStack

  1. Start LocalStack (for example, via Docker):

    docker run --rm -it -p 4566:4566 -p 4510-4559:4510-4559 localstack/localstack

    This exposes the LocalStack APIs on http://localhost:4566 as expected by the provider config.

  2. Initialize Terraform:

    terraform init
  3. Format and validate:

    terraform fmt -check
    terraform validate
  4. Plan and apply against LocalStack:

    terraform plan
    terraform apply

    Confirm with yes when prompted. Terraform will create the S3 bucket in LocalStack rather than AWS; the dummy credentials and endpoint mapping make this safe for local experimentation.

  5. Check outputs:

    terraform output
    terraform output bucket_name
    terraform output account_id
  6. Configure profile

    aws configure --profile localstack
  7. Verify in LocalStack (using AWS CLI configured to point to LocalStack):

    aws --endpoint-url http://localhost:4566 s3 ls --profile localstack

    You should see the bucket named in bucket_name. LocalStack typically uses test credentials and a default account ID of 000000000000.

  8. Destroy (noting prevent_destroy)

    Because of prevent_destroy = true, terraform destroy will refuse to delete the bucket. That’s intentional, to illustrate the lifecycle block. Remove prevent_destroy, run terraform apply again, then:

    terraform destroy

6. A Quick Comparison Table

To solidify the concepts, here is a concise comparison of key block types:

Block type Purpose Typical labels Commonly uses nested blocks
terraform Configure Terraform itself None required_providers, backend
provider Configure connection to an API Provider name (e.g., "aws") Occasionally provider-specific blocks
resource Declare managed infrastructure Resource type, local name lifecycle, provisioner, provider-specific
data Read existing infrastructure Data source type, local name Provider-specific nested blocks
variable Define module inputs Variable name None (just arguments)
output Expose module outputs Output name None (just arguments)
locals Define internal computed values None None (just arguments)

A Practical Workflow for Git Worktrees in Everyday Development

Git worktrees provide a clean, efficient way to work on multiple branches at once without juggling stashes, temporary commits, or extra clones. This article walks through a concrete workflow for adopting git worktree in everyday development.


Why git worktree?

In a typical Git workflow, switching branches while you have uncommitted work forces you to choose between stashing, WIP commits, or creating another clone of the repository. Git worktrees give you another option: multiple checked‑out branches from the same repository, each in its own directory, sharing a single .git data store.

This is especially useful when:

  • You frequently interrupt a feature for hotfixes, reviews, or experiments.
  • Your project has heavy dependencies (node modules, large virtualenvs, Gradle caches), making multiple clones expensive.
  • You want to run tests or builds for several branches in parallel on the same machine.

One‑time setup and conventions

Start from an existing clone of your repository, with main (or develop) checked out:

git worktree list

Initially, you should see only the main worktree (the directory you’re in), showing its path, commit, and branch.

Choose a consistent convention for worktree locations, such as:

  • A .worktrees/ directory inside the main repo (e.g. .worktrees/feature-new-ui).
  • Or sibling directories (e.g. ../project-feature-new-ui).

The important part is that each worktree has a unique, meaningful path and that you avoid nesting one Git worktree inside another, which can confuse Git’s metadata.


Creating a worktree for an existing branch

Suppose your remote has a branch feature/new-ui that you want to work on without leaving main in your primary directory.

From the main repo directory:

git fetch origin
git worktree add .worktrees/new-ui feature/new-ui
cd .worktrees/new-ui

Key points:

  • git worktree add <path> <branch> creates a new directory at <path> and checks out <branch> there.
  • The new directory behaves like a normal working copy: you edit files, run tests, commit, and push as usual.

Your typical flow inside that worktree looks like:

# already in .worktrees/new-ui
git status
# edit files, run tests
git commit -am "Implement new UI"
git push -u origin feature/new-ui

When you’re done for now, you can simply cd back to the main directory, which still has your original branch and working state untouched.


Starting a new feature in its own worktree

Very often, the branch doesn’t exist yet; you want to create it and work in a dedicated directory from the start.

From the main repo directory:

git fetch origin
git worktree add -b feature/new-api .worktrees/new-api origin/main
cd .worktrees/new-api

Here:

  • -b feature/new-api tells Git to create feature/new-api as a new branch.
  • origin/main is the base commit; you can use main, develop, or any other starting point appropriate to your branching model.

Now you can develop the feature completely within .worktrees/new-api, while the main directory remains on main for reviews, builds, or other work.


Managing multiple active worktrees

Over time, you might accumulate several active worktrees: a couple of feature branches, a long‑running refactor, and maybe a release branch.

To see what’s active:

git worktree list

The output lists each worktree’s path, current commit, and checked‑out branch, with the main worktree first. For example:

/home/user/project                     a1b2c3d [main]
/home/user/project/.worktrees/new-ui   d4e5f6 [feature/new-ui]
/home/user/project/.worktrees/new-api  987654 [feature/new-api]

With this view you can:

  • Jump between directories instead of switching branches in a single directory.
  • Keep long‑running work (like big refactors) open and test them periodically without disturbing your day‑to‑day branch.
  • Run multiple test suites or build processes in parallel on different branches.

Each worktree is a self‑contained environment for that branch; there is no “one worktree, many branches” mode—every worktree corresponds to a single branch or detached HEAD at a time.


Handling urgent hotfixes and reviews

A classic use case: you’re mid‑feature when a production incident appears.

Instead of stashing or committing half‑baked work:

  1. Leave your feature worktree as is.

  2. From the main repo directory, create a hotfix worktree:

    git fetch origin
    git worktree add .worktrees/hotfix-critical hotfix/critical-bug
    cd .worktrees/hotfix-critical
  3. Apply the fix, commit, and push:

    # implement fix
    git commit -am "Fix critical bug in production"
    git push origin hotfix/critical-bug
  4. Once the hotfix is merged back into main and any release branches, you can remove this worktree (see next section).

You can use the same pattern for:

  • Checking out a PR branch to test it locally.
  • Pairing on a branch without touching your current environment.
  • Running experiments on a throwaway branch in a dedicated directory.

Cleaning up: remove and prune

Worktrees are cheap, but they will accumulate if you never remove them.

Once a branch is merged and you no longer need its dedicated directory:

# from the main repo directory (or any worktree in the same repo)
git worktree remove .worktrees/new-ui

Important details:

  • git worktree remove <path> removes the worktree directory and its administrative entry; it does not necessarily delete the Git branch itself.
  • The worktree must be clean (no untracked or modified tracked files) unless you add --force, which will discard uncommitted changes.

Over time you may manually delete directories or encounter stale entries (e.g. after a crash). To clean up those leftovers:

git worktree prune --verbose

This command prunes worktree records whose directories no longer exist, using expiration rules that can be configured via Git settings like gc.worktreePruneExpire. You can also use --expire <time> with prune if you want to only remove older, unused entries.

A light maintenance habit is:

  • Remove the worktree for a feature once its PR is merged and the branch is closed.
  • Run git worktree prune occasionally to clean up stale metadata.

Practical guidelines and best practices

To make git worktree a reliable part of your team’s workflow, adopt a few simple rules:

  • Organize worktrees predictably: Use stable directory patterns (.worktrees/<branch-name> or similar) and use names that reflect the branch, like .worktrees/feature-auth-api.
  • Avoid nesting: Never create a worktree inside another worktree’s directory; this can confuse Git’s detection of repositories and worktrees.
  • Keep your base branches fresh: Regularly fetch and update main/develop and rebase or merge them into your feature worktrees to minimize integration surprises.
  • Clean up after merges: Remove worktrees you no longer need, then prune occasionally to ensure git worktree list remains readable and accurate.
  • Check Git version: Some newer options and behaviors (like more detailed list output and improved prune behavior) depend on having a reasonably up‑to‑date Git installation.

By following this workflow—create a worktree per active branch, keep them organized, and clean them up when done—you get parallel branch development with far less friction than stashes, temporary commits, or multiple clones, while still relying on standard Git primitives.

Terraform as Infrastructure as Code

Terraform sits at the center of modern Infrastructure as Code (IaC) practice: we describe infrastructure in text, keep it in Git, and let an engine reconcile desired state with real-world cloud APIs.


Infrastructure as Code (IaC)

Infrastructure as Code is the practice of managing and provisioning infrastructure through machine‑readable configuration files rather than interactive configuration tools or consoles. The crucial mental shift is to treat infrastructure the same way you treat application code: versioned, reviewed, and automated.

Key characteristics:

  • Declarative definitions
    You describe what infrastructure you want (VPCs, subnets, instances, load balancers) instead of scripting how to create it step by step.
  • Version controlled
    Configurations live in Git (or similar), so you get history, diffs, branching, and pull requests for infra changes.
  • Repeatable and consistent
    The same configuration, with different inputs (variables, workspaces), can stand up dev, test, and prod environments that are structurally identical.
  • Testable and reviewable
    Changes are peer‑reviewed, validated via plans, policy checks, and possibly automated tests in CI/CD, instead of ad‑hoc console clicks.

The rationale is straightforward: we already know how to manage complexity in software systems with code and discipline; IaC applies those same practices to infrastructure.


What is Terraform?

Terraform is a declarative Infrastructure as Code tool created by HashiCorp that provisions and manages infrastructure across many platforms (AWS, Azure, GCP, Kubernetes, on‑prem, and various SaaS APIs). You express your desired infrastructure using HashiCorp Configuration Language (HCL), and Terraform figures out the necessary operations to reach that state.

Conceptually, Terraform:

  1. Reads your configuration, which represents the desired state.
  2. Compares it to its state plus the actual infrastructure.
  3. Constructs an execution plan describing the required changes.
  4. Applies the plan by calling provider APIs in the correct dependency order.

Why this model is powerful:

  • Multi‑cloud, single workflow
    The same CLI, syntax, and mental model work across different clouds and services.
  • State‑aware
    Terraform tracks what it has created, so it can safely update or destroy resources without guesswork.
  • Ecosystem and reuse
    A rich registry of modules and providers enables you to stand on others’ shoulders instead of rebuilding common patterns.

In essence, Terraform acts as a reconciliation engine: it continuously aligns reality with the infrastructure state you declare in code.


Core Components of Terraform

While Terraform’s architecture is modular, you mainly interact with a small set of core concepts.

CLI and Core Engine

The CLI (terraform) is your main interface. Terraform Core:

  • Parses HCL configuration files.
  • Builds a dependency graph of resources and data sources.
  • Compares configuration and state to determine what must change.
  • Produces an execution plan and applies it while respecting dependencies.

The dependency graph is central: it ensures, for example, that networks exist before instances are created, and databases exist before applications that depend on them.

Providers

Providers are plugins that encapsulate the logic for talking to external APIs such as AWS, Azure, GCP, Kubernetes, GitHub, Datadog, and many others. Each provider:

  • Defines available resource types and data sources.
  • Implements create, read, update, and delete semantics for those resources.
  • Handles authentication and low‑level API interactions.

The rationale is separation of concerns: Terraform Core stays generic, while providers handle domain‑specific details of each platform.

Resources

Resources are the primitive units of infrastructure in Terraform. Each resource represents a managed object, such as:

  • A virtual machine or container cluster.
  • A network component (VPC, subnet, load balancer).
  • A managed service instance (database, cache, queue).

Terraform manages the full lifecycle of resources: creation, in‑place update when possible, replacement when required, and destruction when no longer desired.

Data Sources

Data sources allow configurations to read information from providers without managing the lifecycle of those objects. They are typically used to:

  • Look up existing infrastructure (e.g., a VPC or subnet created outside the current configuration).
  • Retrieve dynamic values (e.g., the latest AMI matching a filter).
  • Integrate with pre‑existing environments instead of forcing everything to be created by the same codebase.

They keep configurations flexible and reduce hard‑coded values, which improves reuse and maintainability.

State and Backends

Terraform uses a state file to map resources in your configuration to real-world objects in the target platforms. This state:

  • Stores resource identifiers and metadata.
  • Enables Terraform to understand what already exists and what needs to change.
  • Is updated after each successful apply.

Backends determine where this state is stored:

  • Local backend: state in a file on your machine; fine for experiments and learning.
  • Remote backends (e.g., S3, GCS, Terraform Cloud): better for teams, as they support centralized storage, locking, and collaboration.

The rationale is that state is a single source of truth about managed infrastructure, and treating it carefully (remote, locked, backed up) is critical for safe operations.

Modules

Modules are reusable, composable units of Terraform configuration. A module can contain:

  • Resources and data sources.
  • Variables, outputs, locals, and even submodules.

Reasons to structure code into modules:

  • Encapsulation
    Hide low‑level details behind a stable interface of inputs and outputs.
  • Reuse
    Apply the same pattern (e.g., a VPC, a Kubernetes cluster, a standard microservice stack) in multiple environments or projects.
  • Governance
    Centralize best practices and security controls in shared modules, reducing drift and inconsistent patterns across teams.

Variables, Outputs, and Locals

These language constructs support configurability and clarity:

  • Variables
    Declare the inputs your configuration expects (e.g., region, instance type, environment name). They enable per‑environment customization without forking the code.
  • Outputs
    Expose selected values after apply (e.g., IP addresses, ARNs, URLs). Outputs are often consumed by other systems or simply used for manual checks.
  • Locals
    Store computed values or shared expressions to avoid duplication and encode small bits of logic directly in the configuration.

The rationale is to keep your Terraform code DRY, expressive, and easy to reason about as configurations grow.


Typical Terraform Workflow

Terraform promotes a structured workflow that aligns naturally with Git‑based development and CI/CD pipelines. This workflow is crucial to reducing risk and improving predictability.

1. Write Configuration

You start by writing .tf files using HCL:

  • Declare providers and their configuration.
  • Define resources, data sources, modules, variables, locals, and outputs.
  • Organize files logically (root module, submodules, environment dirs).

The focus is on describing the target state of the infrastructure rather than prescribing a sequence of imperative steps.

2. Initialize (terraform init)

You run:

terraform init

This command:

  • Downloads required providers and modules.
  • Sets up the backend for state (local by default, or remote if configured).
  • Prepares the working directory so subsequent commands can function correctly.

Rationale: separating initialization from planning/applying makes dependencies explicit and reproducible, especially across machines or CI runners.

3. Plan (terraform plan)

You run:

terraform plan

Terraform then:

  • Reads configuration and current state.
  • Queries providers for the real infrastructure state.
  • Computes and displays the execution plan, indicating which resources will be created, changed, or destroyed.

The plan serves as your infrastructure “diff,” analogous to git diff for code. In a mature setup, this plan is typically generated as part of a pull request, allowing reviewers to reason about the exact infra impact of a code change before approval.

4. Apply (terraform apply)

You run:

terraform apply

Terraform:

  • Either recomputes or uses a previously saved plan.
  • Executes the required operations in dependency order.
  • Handles partial failures and retries where possible.
  • Updates the state file upon successful completion.

The key practice is discipline: operators avoid ad‑hoc console changes, and instead always modify the .tf files, inspect the plan, and then apply. This ensures the code and reality remain aligned.

5. Destroy (terraform destroy)

When you need to decommission an environment, you run:

terraform destroy

Terraform:

  • Plans the removal of managed resources.
  • Executes deletions in an order that respects dependencies.
  • Updates the state so it no longer references removed resources.

This is especially valuable for ephemeral or per‑branch environments and is a powerful tool for cost control and cleanup.

Setting up a LocalStack VPC with Terraform, Docker Compose, and AWS CLI

1. Overview and rationale

  • LocalStack emulates EC2/VPC APIs locally, so Terraform can create VPCs, subnets, route tables, and gateways just like on AWS.
  • The AWS CLI can talk to LocalStack by using --endpoint-url http://localhost:4566, letting you validate resources with the exact same commands you’d run against real AWS.
  • This keeps your workflow close to production: same Terraform provider, same CLI, different endpoint.

2. Run LocalStack with Docker Compose

Create docker-compose.yml:

version: "3.8"

services:
  localstack:
    image: localstack/localstack:latest
    container_name: localstack
    ports:
      - "4566:4566"              # Edge port: all AWS APIs
      - "4510-4559:4510-4559"    # Optional service ports
    environment:
      - SERVICES=ec2             # Add more: ec2,lambda,rds,ecs,...
      - AWS_DEFAULT_REGION=us-east-1
      - DEBUG=1
      - DOCKER_HOST=unix:///var/run/docker.sock
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"

Start LocalStack:

docker compose up -d

LocalStack now exposes EC2/VPC on http://localhost:4566 (the edge endpoint).


3. Terraform VPC configuration

Keep Terraform AWS‑idiomatic and only change the endpoint.

providers.tf

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region                      = "us-east-1"
  access_key                  = "test"
  secret_key                  = "test"
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true
  s3_use_path_style           = true

  endpoints {
    ec2 = "http://localhost:4566"
  }
}

main.tf

resource "aws_vpc" "demo" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_support   = true
  enable_dns_hostnames = true

  tags = {
    Name = "localstack-demo-vpc"
  }
}

resource "aws_subnet" "public_az1" {
  vpc_id                  = aws_vpc.demo.id
  cidr_block              = "10.0.1.0/24"
  availability_zone       = "us-east-1a"
  map_public_ip_on_launch = true

  tags = {
    Name = "localstack-demo-public-az1"
  }
}

resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.demo.id

  tags = {
    Name = "localstack-demo-igw"
  }
}

resource "aws_route_table" "public" {
  vpc_id = aws_vpc.demo.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }

  tags = {
    Name = "localstack-demo-public-rt"
  }
}

resource "aws_route_table_association" "public_az1_assoc" {
  subnet_id      = aws_subnet.public_az1.id
  route_table_id = aws_route_table.public.id
}

output "vpc_id" {
  value = aws_vpc.demo.id
}

output "public_subnet_id" {
  value = aws_subnet.public_az1.id
}

Apply:

terraform init
terraform apply

This models the classic “public subnet” layout so the same module can later be pointed at real AWS with only a provider change.

4. Configure AWS CLI for LocalStack

You can use the stock AWS CLI v2 and override the endpoint.

  1. Configure a local profile (optional but tidy):
aws configure --profile localstack
# AWS Access Key ID: test
# AWS Secret Access Key: test
# Default region name: us-east-1
# Default output format: json
  1. For each command, add:
--endpoint-url http://localhost:4566 --profile localstack

This tells the CLI to send EC2 API calls to LocalStack instead of AWS.

If you prefer less typing, you can define an alias (e.g. in your shell):

alias awslocal='aws --endpoint-url http://localhost:4566 --profile localstack'

This is conceptually the same as the awslocal wrapper LocalStack provides, but you stay completely within standard AWS CLI semantics.


5. Validating the VPC with AWS CLI

After terraform apply, use the CLI to verify each piece of the VPC.

5.1 List VPCs

aws ec2 describe-vpcs --endpoint-url http://localhost:4566 --profile localstack

or with the alias:

awslocal ec2 describe-vpcs

Check for a VPC with:

  • CidrBlock = 10.0.0.0/16
  • Tag Name = localstack-demo-vpc

The shape of the output is identical to real AWS describe-vpcs.

5.2 List subnets

awslocal ec2 describe-subnets

Confirm a subnet exists with:

  • CidrBlock = 10.0.1.0/24
  • AvailabilityZone = us-east-1a
  • Tag Name = localstack-demo-public-az1

5.3 List Internet Gateways

awslocal ec2 describe-internet-gateways

You should see an IGW whose Attachments includes your VPC ID and has tag localstack-demo-igw.

5.4 List route tables

awslocal ec2 describe-route-tables

Verify:

  • A route table tagged localstack-demo-public-rt.
  • A route with DestinationCidrBlock 0.0.0.0/0 and GatewayId set to your IGW ID.
  • An association to your public subnet ID.

These commands mirror the official AWS CLI usage for EC2, just with the endpoint overridden.

6. Why this testing approach scales

  • Uses only Terraform + AWS CLI, tools you already depend on in real environments.
  • Easy to script: you can wrap the CLI checks into bash scripts or CI jobs to assert your VPC configuration in LocalStack before promoting to AWS.
  • Mental model stays aligned with production AWS: same commands, same JSON structures, just a different base URL.

Running WordPress and MariaDB with Docker Compose

Running WordPress with Docker Compose gives you a reproducible, portable environment that is easy to spin up for local development or small deployments. In this article, we will build a minimal but robust setup that uses WordPress with MariaDB, including persistent storage and a proper health check using MariaDB’s built-in healthcheck.sh script.

Overview of the architecture

Our stack consists of two containers on the same Docker network: a MariaDB container as the database and a WordPress container running Apache and PHP 8.2. We add named volumes for persistence, environment variables for configuration, and a health check on MariaDB so WordPress waits for a fully initialized InnoDB engine before attempting to connect.

The docker-compose.yml

Below is the full docker-compose.yml that works with current MariaDB and WordPress images:

version: "3.9"

services:
  db:
    image: mariadb:11.3
    container_name: wordpress-db
    restart: unless-stopped
    environment:
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: wordpress_password
      MYSQL_ROOT_PASSWORD: root_password
    volumes:
      - db_data:/var/lib/mysql
    healthcheck:
      test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 60s

  wordpress:
    image: wordpress:php8.2-apache
    container_name: wordpress-app
    depends_on:
      db:
        condition: service_healthy
    restart: unless-stopped
    environment:
      WORDPRESS_DB_HOST: db:3306
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: wordpress_password
      WORDPRESS_DB_NAME: wordpress
    ports:
      - "8080:80"
    volumes:
      - wordpress_data:/var/www/html

volumes:
  db_data:
  wordpress_data:

This configuration keeps the database and WordPress state in named volumes and wires WordPress to wait for a healthy MariaDB instance using the official health check script.

Service: MariaDB (db)

The db service defines the MariaDB database container and uses the official MariaDB image with its integrated health check script.

Key aspects:

  • image: mariadb:11.3 pins a recent stable MariaDB release, which helps keep behavior consistent and avoids surprises from latest. environment sets up the initial database: MYSQL_DATABASE, MYSQL_USER, MYSQL_PASSWORD, and MYSQL_ROOT_PASSWORD are read by the entrypoint to create the schema and users on first run.
  • volumes: - db_data:/var/lib/mysql stores the MariaDB data directory in a named volume so your data persists across container recreations and upgrades.

The health check is where this service becomes more robust:

healthcheck:
  test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
  interval: 10s
  timeout: 5s
  retries: 5
  start_period: 60s
  • healthcheck.sh is a script shipped in the MariaDB official Docker image specifically for health monitoring and supports multiple tests.
  • --connect verifies that a client can successfully connect to the server using the dedicated healthcheck user created by the image.
  • --innodb_initialized ensures that the InnoDB storage engine has finished initialization, preventing false positives during the initial database bootstrap.
  • start_period: 60s gives MariaDB extra time to initialize before failed checks count against the retries limit, which is useful for first startup on larger or slower disks.

This approach is more accurate than using mysqladmin ping, which can report success even while initialization or upgrade routines are still running.

Service: WordPress (wordpress)

The wordpress service defines the application container that runs WordPress on Apache with PHP 8.2.

Important points:

  • image: wordpress:php8.2-apache uses the official WordPress image variant that bundles Apache and PHP 8.2, giving you a maintained, up-to-date runtime.
  • depends_on: db: condition: service_healthy tells Docker Compose not just to start containers in order, but to wait until MariaDB’s health check passes, which avoids race conditions at startup.
  • The database environment variables (WORDPRESS_DB_HOST, WORDPRESS_DB_USER, WORDPRESS_DB_PASSWORD, WORDPRESS_DB_NAME) mirror the MariaDB configuration and instruct WordPress how to connect to the database over the internal Docker network.
  • ports: "8080:80" publishes WordPress on port 8080 of your host so you can access it at http://localhost:8080.
  • volumes: - wordpress_data:/var/www/html keeps the WordPress core, plugins, themes, and uploads persistent in a named volume, which protects content and configuration from container lifecycle changes.

This container design keeps WordPress stateless from the perspective of the image and pushes state into Docker-managed volumes, which aligns well with container best practices.

Volumes: persistence layer

The volumes section declares two named volumes used by the services:

volumes:
  db_data:
  wordpress_data:
  • db_data holds the MariaDB data directory, which contains all databases defined by your instance.
  • wordpress_data holds the WordPress application files and uploaded media, including themes and plugins.

Named volumes are managed by Docker and are not removed by a plain docker compose down, which means your data survives service restarts and configuration tweaks. If you explicitly run docker compose down --volumes, Docker will remove these volumes and you will lose the database and WordPress content.

Running and managing the stack

To bring this stack up:

  1. Create a directory (for example, wordpress-stack) and save the YAML above as docker-compose.yml in it.

  2. Edit the environment values for passwords and database names to suit your environment and security policies, ideally moving them into an .env file.

  3. From that directory, start the stack in detached mode:

    docker compose up -d
  4. Wait for the containers to reach a healthy state, then open:

    http://localhost:8080

    and complete the WordPress installation wizard.

To stop the stack while preserving data:

docker compose down

To completely tear down including volumes and stored data:

docker compose down --volumes

This workflow gives you a repeatable, self-contained WordPress + MariaDB environment that starts reliably, thanks to the health-checked database service.

Simple Terraform Config to Setup AWS S3 Sandbox in LocalStack

This article shows how to run a local AWS‑like S3 environment with LocalStack in Docker, manage buckets with Terraform, and inspect everything visually using an S3 GUI client such as S3 Browser (or any S3‑compatible desktop app).


1. Overview of the setup

You will end up with:

  • LocalStack running via docker-compose.yml, exposing S3 on http://localhost:4566.
  • Terraform creating an S3 bucket, enabling versioning, and adding a lifecycle rule.
  • S3 Browser (or a similar S3 GUI) connected to LocalStack so you can see buckets and object versions visually.

Rationale: this mirrors a real AWS workflow (Infra as Code + GUI) while remaining entirely local and safe to experiment with.


2. LocalStack with docker-compose.yml

Create a working directory, e.g. localstack-s3-terraform, and add docker-compose.yml:

version: "3.8"

services:
  localstack:
    image: localstack/localstack:latest
    container_name: localstack
    ports:
      - "4566:4566"          # Edge port: all services, including S3
      - "4510-4559:4510-4559"
    environment:
      - SERVICES=s3          # Only start S3 for this demo
      - DEBUG=1
      - DOCKER_HOST=unix:///var/run/docker.sock
    volumes:
      - "./localstack-data:/var/lib/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"

Key aspects:

  • Port 4566 is the single “edge” endpoint for S3 and other services in current LocalStack.
  • SERVICES=s3 keeps the environment focused and startup fast.
  • ./localstack-data persists LocalStack state (buckets and objects) between restarts.

Start LocalStack:

docker compose up -d

3. Terraform config with versioning and lifecycle

In the same directory, create main.tf containing the AWS provider configured for LocalStack and S3 with versioning + lifecycle policy:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region                      = "ap-southeast-2"
  access_key                  = "test"
  secret_key                  = "test"
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true
  s3_use_path_style           = true

  endpoints {
    s3 = "http://localhost:4566"
  }
}

resource "aws_s3_bucket" "demo" {
  bucket = "demo-bucket-localstack"
}

resource "aws_s3_bucket_versioning" "demo_versioning" {
  bucket = aws_s3_bucket.demo.id

  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_lifecycle_configuration" "demo_lifecycle" {
  bucket = aws_s3_bucket.demo.id

  rule {
    id     = "expire-noncurrent-30-days"
    status = "Enabled"

    filter {
      prefix = "" # apply to all objects
    }

    noncurrent_version_expiration {
      noncurrent_days = 30
    }
  }
}

Important Terraform points:

  • Provider: points to http://localhost:4566 so all S3 calls go to LocalStack, not AWS.
  • Dummy credentials (test / test) are sufficient; LocalStack doesn’t validate real AWS keys.
  • Versioning is modeled as a separate resource to clearly express bucket behavior.
  • Lifecycle configuration is modeled explicitly as well, aligning with AWS best practices and lifecycle examples.

Initialize and apply:

terraform init
terraform apply

Confirm when prompted; Terraform will create the bucket, enable versioning, and attach the lifecycle rule.


4. Configuring S3 Browser (or similar GUI) for LocalStack

Now that LocalStack is running and Terraform has created your bucket, you connect S3 Browser (or any S3 GUI) to LocalStack instead of AWS.

In S3 Browser, create a new account/profile with something like:

  • Account name: LocalStack (any label you like).
  • S3 endpoint / server: http://localhost:4566
  • Access key: test
  • Secret key: test
  • Region: ap-southeast-2

Make sure your client is configured to use the custom endpoint instead of the standard AWS endpoints (this is usually done in an “S3 Compatible Storage” as the Account Type).

Once saved and connected:

  • You should see the bucket demo-bucket-localstack in the bucket list.
  • Opening the bucket lets you upload, delete, and browse objects, just as if you were talking to real S3.
« Older posts