AWS - Ephemeral CI/CD Infrastructure with Terraform - Ansible - GitHub Actions

Inception

Hello everyone, today blog is about ephemeral infrastructure and how we can use the DevOps tools to build temporary infrastructure for dev testing. It can be performed by mixing the benefits of Infrastructure as code concepts and automations like Terraform, Ansible, and GitHub Actions. I will explain each concept as per and other sub tool concept tool.

Overview

This task demonstrates the core idea of using Terraform and Ansible in a pipeline workflow. Every push to the main branch triggers the pipeline to test the developer's code; in our scenario, it is an index.html file. When developer push a code to the main, Terraform is triggered, then confirms with its portal about the state file statuses, check for AWS authentication, and after that it decides to deploy or destroy. The authentication between HCP cloud and AWS is with OpenID Connect for short-lived credentials assumed by Terraform to be able to deploy from the CI runner. After provisioning, the Ansible role comes and takes the generated public key to SSH on the EC2 provisioned. The public IP of the EC2 is passed in the CI runner by pipeline logic using the ouput.tf file to generate a dynamic inventory.ini file, and then ansible run the playbook.yaml and configure the EC2 by downloading nginx, and then copies the index.html file to the default path of nginx. After that, a validation was performed to ensure that everything works correctly. Last job is a TTL count for 2 hours or any time you predict your test will finish, after this TTL, a destroy job starts. This demonstrates that every tool has its own role and job. Terraform is responsible for provisioning infrastructure; on the other hand, Ansible is about infrastructure configuration management while the pipeline is the daemon that drive all this workflow in automated way. Before we start the implementation steps, there some concepts you should revise before attempting.

What is Terraform?

Terraform is an Infrastructure as Code (IaC) tool that lets you define, provision, and manage infrastructure declaratively using configuration files, instead of manually creating resources in a cloud console. You describe the desired end state of your infrastructure (for example: networks, compute, load balancers, databases), and Terraform figures out how to reach and maintain that state by creating, updating, or deleting resources through provider APIs such as AWS, Azure, or GCP.

You can read about Terraform and more here in this Terraform series. It’s very straightforward:

Terraform Basics Essentials

What is Ansible?

Ansible is a configuration management and automation tool used to configure, manage, and maintain systems after they are provisioned. Instead of defining infrastructure resources, Ansible focuses on what should be installed, configured, and running on servers—such as packages, services, files, users, and application settings. You describe this desired system state in YAML playbooks, and Ansible ensures machines match that state.

Ansible is agentless, meaning it does not require any software installed on target machines; it connects remotely (most commonly via SSH) and executes tasks using built-in modules. This makes it simple, secure, and easy to integrate into CI/CD pipelines. In modern workflows, Ansible is often used after Terraform: Terraform provisions the infrastructure (VMs, networks), and Ansible configures those VMs (installing Nginx, deploying applications, setting system configs). This clear separation makes infrastructure reproducible, configuration consistent, and operations easier to automate and scale.

What is GitHub Actions?

GitHub Actions is a CI/CD and automation platform built directly into GitHub that allows you to automatically run workflows in response to events in a repository, such as pushing code, opening a pull request, or merging to a branch. These workflows are defined as YAML files and are executed by runners (hosted by GitHub or self-hosted) that perform tasks like building code, running tests, provisioning infrastructure, or deploying applications.

In practice, GitHub Actions acts as the execution engine of modern DevOps pipelines: it orchestrates tools such as Terraform, Ansible, Docker, and cloud CLIs in a controlled, repeatable way. Because workflows are versioned alongside the code, GitHub Actions enables traceable, auditable automation where every infrastructure or deployment change is tied to a commit. This makes it especially effective for infrastructure and platform workflows, where GitHub Actions triggers the pipeline, runs automation tools, and enforces consistent processes without manual intervention.

What is Terraform Cloud?

Terraform Cloud, also known as HCP Terraform, is a managed service by HashiCorp that provides a central control plane for Terraform operations. It is not a replacement for Terraform itself, but a platform that enhances Terraform by managing remote state, state locking, variables, credentials, and run coordination in a secure and collaborative way. Instead of storing Terraform state locally or in ad-hoc backends, Terraform Cloud keeps state centrally, ensuring consistency, preventing conflicts, and enabling safe team workflows.

Beyond state management, Terraform Cloud adds identity, security, and governance capabilities to Terraform workflows. It can broker cloud credentials using mechanisms like OIDC, enforce policies, provide audit logs, and coordinate runs triggered by CI systems, APIs, or version control systems. Whether Terraform is executed locally, in CI, or remotely within the platform, Terraform Cloud acts as the source of truth and trust for infrastructure changes, making Terraform usage more secure, scalable, and production-ready for teams and organizations.

What is OIDC?

OIDC (OpenID Connect) is an identity federation protocol built on top of OAuth 2.0 that allows a system to prove its identity using short-lived, verifiable tokens instead of long-term usernames and secrets. In cloud and CI/CD contexts, OIDC enables external workloads—such as CI pipelines, Kubernetes clusters, or infrastructure tools—to authenticate to cloud providers by presenting a signed identity token (JWT) issued by a trusted identity provider, rather than storing static credentials.

When used with platforms like AWS, OIDC allows the cloud provider to trust an external identity provider, validate the token’s claims (who issued it, who is requesting access, and for what purpose), and then issue temporary, scoped credentials via its security service (for example, AWS STS). This model eliminates long-lived secrets, improves security, and centralizes access control. As a result, OIDC is now the recommended authentication mechanism for modern cloud automation, replacing hard-coded credentials in CI/CD pipelines and infrastructure tooling.

Workflow diagram

Implementation Steps

• Create identity provider for Terraform

  • In AWS Console:

    • select IAM

    • click Identity provider

    • select add provider

    • Choose OpenID Connect

    • Then specify the secure OpenID Connect URL for authentication requests. ex: github.com, app.terraform.io

    • In Audience, specify the client ID issued by the Identity provider for your app. ex: aws.workload.identity, sts.amazonaws.com

    • Select Add provider

• Create Role

  • In IAM select Role

  • Create Role

  • Choose the Trusted entity type: Web identity

  • choose the provider URL you created before and its Audience

  • Choose Workload type: Work Space Run

  • Then specify the Organization, Workspace, and maybe like this *

⚠️ This is in case you need to make all Terraform runs and limit AWS access to ONLY a specified workspace within that Organization, so take care.

• Then choose those 3 policy names:

  • AmazonEC2FullAccess

  • AmazonSSMFullAccess

  • AmazonVPCFullAccess

• Name the role hcp-terraform-aws-role, then click Create Role

✅ AWS now trusts Terraform Cloud as an identity provider, and limits AWS access to ONLY that workspace.

• Configure Terraform Cloud Workspace

  • Open Terraform Cloud

  • Open workspace

  • Go to Settings, then Variables

  • Add these Environment Variables

    • Name: TFC_AWS_PROVIDER_AUTH

    • value: true

    • Not Sensitive

    • Name: TFC_AWS_RUN_ROLE_ARN

    • value: arn:aws:iam::<ACCOUNT_ID>:role/hcp-terraform-aws-role

    • Not Sensitive

    • Name: AWS_REGION

    • value: us-east-1

    • Not Sensitive

⚠️ Category must be Environment, not Terraform

• GitHub Actions secrets

  • In HCP Terraform Console, select account settings

  • Select Tokens

  • Click on Create an API Token, then copy it

  • In GitHub, Select Settings

  • In the left menu, choose Secrets and variables, Actions

  • In Repository secrets, click on New repository secret

  • Add TF_API_TOKEN then paste the token

  • Add another secret SSH_PRIVATE_KEY, then paste the ssh key you generated

• Make the file structure like this:

• Terraform configuration

  • main.tf

provider "aws" {}

• variables.tf

variable "region" { description = "AWS region" }

variable "key_name" { type = string }

variable "public_key" { type = string }

variable "instance_type" { default = "t3.micro" }

variable "ami_id" { description = "The AMI ID for the NGINX server" default = "ami-068c0051b15cdb816" }

• auth.tf

  
    resource "aws_key_pair" "ci_key" {
    key_name   = var.key_name
    public_key = var.public_key
  
    }
  

• ec2.tf

    resource "aws_instance" "web" {
    ami = var.ami_id
    instance_type = var.instance_type
    key_name = var.key_name
    vpc_security_group_ids = [aws_security_group.web_sg.id]
  
    tags = {
      Name = "ci_ephemeral_web"
  
    associate_public_ip_address = true  
    }
  
    }
  

• sg.tf

  ```go resource "aws_security_group" "web_sg" {

  name = "public-ec2-sg"

  ingress { from_port = 22 to_port = 22 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] }

ingress { from_port = 80 to_port = 80 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] }

egress { from_port = 0 to_port = 0 protocol = -1 cidr_blocks = ["0.0.0.0/0"] }

}

* output.tf

    ```go
    output "public_ip" {
      value = aws_instance.web.public_ip


    }

• Ansible Configuration

  • playbook.yml

    - hosts: web
    become: true
    tasks:
      - name: Install nginx
        yum:
          name: nginx
          state: present
          update_cache: yes

      - name: Copy web content
        copy:
          src: index.html
          dest: /usr/share/nginx/html
          mode: '0644'

      - name: Start nginx
        service:
          name: nginx
          state: started
          enabled: true

• Github Action logic

    name: Ephemeral Infra CI/CD (Terraform Cloud + Ansible)

    on:
      push:
        branches:
          - main

    env:
      TF_CLOUD_ORG: aws_pipelines
      TF_WORKSPACE: aws_dev
      TF_VAR_region: ${{ secrets.AWS_REGION }}
      TF_VAR_key_name: ${{ secrets.KEY_NAME }}

    jobs:
      # --------------------------------------------------
      # JOB 1: Terraform Init
      # --------------------------------------------------
      terraform_init:
        runs-on: ubuntu-latest

      steps:
        - name: Checkout repository
          uses: actions/checkout@v4

        - name: Setup Terraform
          uses: hashicorp/setup-terraform@v3

        - name: Authenticate to Terraform Cloud
          run: |
            echo 'credentials "app.terraform.io" {
              token = "${{ secrets.TF_API_TOKEN }}"
            }' > ~/.terraformrc

        - name: Terraform Init
          run: |
            cd terraform
            terraform init


    # --------------------------------------------------
    # JOB 2: Terraform Plan
    # --------------------------------------------------
    terraform_plan:
      needs: terraform_init
      runs-on: ubuntu-latest

      steps:
        - name: Checkout repository
          uses: actions/checkout@v4

        - name: Create SSH key files
          run: |
            echo "${{ secrets.SSH_PRIVATE_KEY }}" > key.pem
            chmod 600 key.pem
            ssh-keygen -y -f key.pem > key.pem.pub


        - name: Export public key to Terraform
          run: |
            echo "TF_VAR_public_key=$(cat key.pem.pub)" >> $GITHUB_ENV
            echo "TF_VAR_key_name=ci-key" >> $GITHUB_ENV


        - name: Setup Terraform
          uses: hashicorp/setup-terraform@v3

        - name: Authenticate to Terraform Cloud
          run: |
            echo 'credentials "app.terraform.io" {
              token = "${{ secrets.TF_API_TOKEN }}"
            }' > ~/.terraformrc

        - name: Terraform Init
          run: |
            cd terraform
            terraform init

        - name: Terraform Plan
          run: |
            cd terraform
            terraform plan

    # --------------------------------------------------
    # JOB 3: Terraform Apply
    # --------------------------------------------------
    terraform_apply:
      needs: terraform_plan
      runs-on: ubuntu-latest
      outputs:
        ec2_ip: ${{ steps.tf_output.outputs.ec2_ip }}

      steps:
        - name: Checkout repository
          uses: actions/checkout@v4

        - name: Create SSH key files
          run: |
            echo "${{ secrets.SSH_PRIVATE_KEY }}" > key.pem
            chmod 600 key.pem
            ssh-keygen -y -f key.pem > key.pem.pub


        - name: Export public key to Terraform
          run: |
            echo "TF_VAR_public_key=$(cat key.pem.pub)" >> $GITHUB_ENV
            echo "TF_VAR_key_name=ci-key" >> $GITHUB_ENV


        - name: Setup Terraform
          uses: hashicorp/setup-terraform@v3

        - name: Authenticate to Terraform Cloud
          run: |
            echo 'credentials "app.terraform.io" {
              token = "${{ secrets.TF_API_TOKEN }}"
            }' > ~/.terraformrc

        - name: Terraform Init
          run: |
            cd terraform
            terraform init

        - name: Terraform Apply (from plan)
          run: |
            cd terraform
            terraform apply -auto-approve

        - name: Export EC2 Public IP
          id: tf_output
          run: |
            echo "ec2_ip=$(terraform -chdir=terraform output -raw public_ip)" >> $GITHUB_OUTPUT


    # --------------------------------------------------
    # JOB 4: Ansible Configuration
    # --------------------------------------------------
    ansible_configure:
      needs: terraform_apply
      runs-on: ubuntu-latest

      steps:
        - name: Checkout repository
          uses: actions/checkout@v4

        - name: Create SSH key
          run: |
            echo "${{ secrets.SSH_PRIVATE_KEY }}" > key.pem
            chmod 600 key.pem

        - name: Generate dynamic inventory
          run: |
            echo "[web]" > inventory.ini
            echo "${{ needs.terraform_apply.outputs.ec2_ip }} ansible_user=ec2-user ansible_ssh_private_key_file=key.pem" >> inventory.ini

        - name: Run Ansible playbook
          run: |
            ansible-playbook \
            -i inventory.ini ansible/playbook.yml \
            --ssh-extra-args="-o StrictHostKeyChecking=no"


    # --------------------------------------------------
    # JOB 5: Validate Application
    # --------------------------------------------------
    validate_app:
      needs: [terraform_apply, ansible_configure]
      runs-on: ubuntu-latest

      steps:
        - name: Validate NGINX endpoint
          run: |
            curl -f http://${{ needs.terraform_apply.outputs.ec2_ip }}


    # --------------------------------------------------
    # JOB 6: TTL Destroy (Always Runs)
    # --------------------------------------------------
    ttl_destroy:
      needs: [terraform_apply, validate_app]
      runs-on: ubuntu-latest
      if: always()

      steps:
        - name: Checkout repository
          uses: actions/checkout@v4


        - name: Setup Terraform
          uses: hashicorp/setup-terraform@v3

        - name: Authenticate to Terraform Cloud
          run: |
            echo 'credentials "app.terraform.io" {
              token = "${{ secrets.TF_API_TOKEN }}"
            }' > ~/.terraformrc

        - name: Recreate SSH key files
          run: |
            echo "${{ secrets.SSH_PRIVATE_KEY }}" > key.pem
            chmod 600 key.pem
            ssh-keygen -y -f key.pem > key.pem.pub

        - name: Export Terraform variables
          run: |
            echo "TF_VAR_key_name=ci-key" >> $GITHUB_ENV
            echo "TF_VAR_public_key=$(cat key.pem.pub)" >> $GITHUB_ENV
            echo "TF_VAR_region=us-east-1" >> $GITHUB_ENV

        - name: Wait for TTL (10 mins)
          run: sleep 600

        - name: Terraform Destroy
          run: |
            cd terraform
            terraform init
            terraform destroy -auto-approve

References

• Create a role for OpenID Connect federation (console)

• Terraform + AWS