Automating OpenTofu GitHub Actions with Workflow Dispatch

Automating OpenTofu GitHub Actions with Workflow Dispatch

As the infrastructure at any given company begins to scale, the complexity that goes along with it grows in a similar parity to where many of the processes and jobs that underpin the platform begin to struggle under the weight of the growing load.

Without a way to address this, the potential for service outages due to lack of durability in processes and functions can impact growth and user experience. Implementing this durability is a lot harder in practice than in theory however, and often requires a large amount of infrastructural black magic to ensure multi-tenancy and scalability. In this post we’re going to present a use case using OpenTofu, GitHub Actions, and workflow-dispatch triggers.

Why We Chose GitHub Actions and Workflow Dispatch

When used efficiently, GitHub Actions can be a powerful resource for automating the orchestration and provisioning of applications on the fly in a declarative fashion using GitOps. As organizations change over time, the need for dynamic provisioning while also maintaining declarative infrastructure also increases.

With GitHub Actions, you can set the parameters of your infrastructure in set a way that you know you will get a repeatable provision each time you trigger the action and it succeeds. This is where the benefit of triggers such as workflow-dispatch come into play.

There’s often a misconception that GitHub Actions can only be triggered by things like a PR or a new commit to a repo, which doesn’t fit many use cases such as cloud self-service. With workflow-dispatch you can dynamically trigger a GitHub Action via the GitHub REST API. For example, let’s say we wanted to set up a self-service process for Kimchi, a new hosted durable functions platform we want to build using Inngest. With a workflow-dispatch trigger and OpenTofu, this is totally possible.

Creating a Workflow Dispatch Trigger

To see this in practice, we’re going to create a GitHub Action using workflow-dispatch and OpenTofu to provision Inngest for durable long-living functions. To get started, we’re going to want to create a .github/workflows/.

To get started, we’re going to want to create a inngest-deploy.yaml file in a .github/workflows directory on our kimchi-infra repo.

In this repo, we’re going to have a directory structure something like this:

├── .github
│   └── workflows
│       └── inngest-deploy.yaml
├── .opentofu
├── main.tf
├── variables.tf
└── README.md

In our inngest-deploy.yaml file, we’re going to want to define a few things:

• The name of the workflow
• The trigger for the workflow (in this case, workflow-dispatch)
• The inputs for the workflow (in this case, tenant_id and tenant_name)
• The jobs for the workflow (in this case, provision)
• The steps for the job (in this case, checkout, setup-terraform, terraform-init, terraform-apply)

Create OpenTofu Files

Before we dive into creating our inngest-deploy.yaml file, we’re going to want to create our OpenTofu configurations. In this case, we’re going to want to create a main.tf and variables.tf file in a hypothetical .opentofu directory. In this directory, we’re going to want to define a few things:

• The provider for our infrastructure (in this case, AWS)
• The resources we want to provision (in this case, an Inngest function)
• The variables we want to use in our infrastructure (in this case, tenant_id and tenant_name)

Here’s an example implementation of how the main.tf could look:

# Example OpenTofu configuration to provision an EC2 instance on AWS

provider "aws" {
  region = "us-east-1"  # Replace with your desired region
}

resource "aws_instance" "inngest_instance" {
  ami           = "ami-xxxxxxxxxxxxxxxxx"  # Replace with a valid AMI for your environment
  instance_type = "t2.micro"  # Choose your desired instance type

  tags = {
    Name = "InngestInstance"
  }

  user_data = <<-EOF
              #!/bin/bash
              # Install and start Inngest
              curl -fsSL https://downloads.inngest.com/inngest-linux-amd64.tar.gz -o /tmp/inngest.tar.gz
              tar -xzvf /tmp/inngest.tar.gz -C /tmp
              /tmp/inngest start  # Or use any other commands for setting up Inngest
              EOF
}

output "instance_ip" {
  value = aws_instance.inngest_instance.public_ip
}

Setting up the Action

Now that we have our OpenTofu configs, we’re going to want to create our inngest-deploy.yaml file. This file will be responsible for defining the workflow that will be triggered when we call the GitHub REST API.

name: Provision Inngest with OpenTofu

on:
  workflow_dispatch:  # Trigger manually from GitHub UI

jobs:
  setup:
    runs-on: ubuntu-latest  # Run on a GitHub-hosted runner

    steps:
    - name: Checkout repository
      uses: actions/checkout@v3

    - name: Set up OpenTofu
      run: |
        curl -sSL https://github.com/opentofu/opentofu/releases/download/v1.8.5/tofu_1.8.5_linux_amd64.tar.gz -o /tmp/tofu.tar.gz
        tar -xzf /tmp/tofu.tar.gz -C /tmp
        mv /tmp/tofu /usr/local/bin/tofu
        chmod +x /usr/local/bin/tofu
        tofu --version  # Verify the installation

    - name: Set up AWS credentials
      uses: aws-actions/configure-aws-credentials@v4
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: ${{ secrets.AWS_REGION }}

    - name: Initialize OpenTofu configuration
      run: |
        # Initialize OpenTofu configuration
        tofu init -input=true

    - name: Apply OpenTofu configuration to provision Inngest
      run: |
        # Run OpenTofu to apply configuration and provision resources
        tofu apply -auto-approve

    - name: Verify Inngest deployment
      run: |
        # Add steps here to verify that Inngest has been provisioned
        # This might include checking the EC2 instance status, verifying logs, etc.
        curl -s http://<YOUR_INNGEST_INSTANCE_IP>:<PORT>/status  # Replace with actual IP/Port to verify Inngest

Triggering the GitHub Action

Now that we’ve set up the framework for action, we’ll want to run a git push so we can trigger it via the REST API. GitHub provides a few options for triggering Actions via workflow-dispatch through the REST API such as JavaScript or cURL.

Option 1. JavaScript via Octokit

We can create a workflow-dispatch event via JavaScript using Octokit by initializing the client and providing the request with the POST method of your repo’s dispatches endpoint:

// https://github.com/octokit/core.js#readme
const octokit = new Octokit({
  auth: 'YOUR-TOKEN'
})

await octokit.request('POST /repos/{owner}/{repo}/actions/workflows/{workflow_id}/dispatches', {
  owner: 'OWNER',
  repo: 'REPO',
  workflow_id: 'WORKFLOW_ID',
  ref: 'topic-branch',
  inputs: {
    name: 'Mona the Octocat',
    home: 'San Francisco, CA'
  },
  headers: {
    'X-GitHub-Api-Version': '2022-11-28'
  }
})

Option 2. cURL requests

Alternatively, you can make generic cURL requests to https://api.github.com/repos/OWNER/REPO/actions/workflows/WORKFLOW_ID/dispatches and provide your bearer auth token to trigger dispatches the same way as above. This would also work with fetch assuming you don’t want to use Octokit.

To find the WORKFLOW_ID, navigate to the Actions tab in your repo and find the number on the end of the /runs/ path. Alternatively, you can pass the workflow file name.

Conclusion

The use case of multi-tenant automated deployments is an often nebulous and complex infrastructural challenge that many startups and even large enterprises face as they adopt new methods of deployment and self-service. One of the benefits of OpenTofu over the commercialized Terraform is that you can deploy without relying on services like the HCP and by taking advantage of open-source forks of Terraform that are community-led in their development.

When OpenTofu and GitHub Actions workflow_dispatch isn’t enough, you can use services like Terrateam to handle infrastructure orchestration and deployments using GitHub Actions with familiar workflows.