How Terrateam Works

Terrateam is a GitHub application that translates GitHub events into Terraform executions. There are two major components of the Terrateam service:

• The backend which receives GitHub events and makes decisions using the event payload
• The Terrateam runner which executes the jobs that the backend creates

Execution Steps

1. A user performs an operation, such as opening a pull request, closing one, or commenting on one. This results in an event being sent to the Terrateam backend.
2. The Terrateam backend evaluates the event to determine if the GitHub Action should be run. If there are changes that require a Terraform run, the backend triggers a GitHub Action workflow dispatch.
3. The Terrateam GitHub Action executes the required work, generally running a Terraform operation in each directory that has changed. Once complete, it publishes the results back to the Terrateam backend.
4. Finally, the Terrateam backend publishes any results back to the pull request in the form of a comment as well as updating any status checks.

Operations

There are two Terraform operations that can be executed with Terrateam:

• Plan
• Apply

Terrateam references these operations internally as the following:

• Plan
• Autoplan
• Apply
• Autoapply

The distinction between a plan and apply are initiated manually by a customer and the auto versions are initiated indirectly due to a customer action such as creating a change, pushing an update to a change, or merging a change.

Plan

Plans have the following semantics (per repository):

1. Multiple plan operations can be executing concurrently.
2. A plan cannot be executing if an apply is executing.
3. A plan can be repeatedly executed.

Apply

Applies have the following semantics (per repository):

1. An apply has higher priority than a plan.
2. An apply is mutually exclusive with all other operations.
3. All changes in the apply must have a plan on the change id.
4. The most recent plan for all changes that will be applied must be successful.
5. The change must either have no other applies, or be explicitly unlocked, or any applied changes must be merged into the default branch.

Event Evaluation

To evaluate a GitHub event, one of the following must be true:

1. Pull request open
2. Pull request sync
3. Pull request close
4. Issue comment
   1. Comment is terrateam plan ...
   2. Comment is terrateam apply ...

If the event corresponds to a possible work manifest, the pull request is then evaluated by the Terrateam backend.

Executing a GitHub Actions Workflow

Receiving Events

The events the Terrateam backend operates on:

1. Installation event:
   1. Create
   2. Suspend
   3. Resume
   4. Uninstall
2. Update to pull request:
   1. Create a pull request
   2. Synchronize a pull request (updating it)
   3. Closing a pull request (either by merge or just closing)
3. Commenting:
   1. terrateam plan - Execute a plan.
   2. terrateam apply - Execute an apply.

In the case of event types (2) and (3), we will construct a work manifest which may be executed. The output of this step is what pull request to operate on and if it is a manual or automatic operation.

Evaluating a Pull Request and Enqueuing

Terrateam will fetch the current state of the PR and collect the following information:

1. The state of the PR: open, closed, merged.
2. The destination branch.
3. If the destination branch is the default_branch.
4. The base_sha and the sha of the PR.
5. The (dir, workspace) list of changed files between the sha’s.
6. The (dir, workspace) list of changed files when applying run_type and tag_query.

Executing the Work Manifest

After any change in state, the Terrateam backend initiates the execution of any work manifests by performing the following steps:

1. Select all work manifests that are queued
2. For each work manifest, group by repository and rank based on priority and creation date.
   1. autoapply and apply have higher priority than autoplan and plan.
   2. Creation date is ordered oldest to newest.
3. Sort all work manifests by creation date and take the top ranked work manifest for each repository
4. Skip any rows that are being locked by a concurrent update.
5. Limit results to the first row.
6. Set state of work manifest to running.

Locking

Terrateam ensures that changes are safely applied. The backend guarantees the following:

1. If an apply is executing on a (repository, directory, workspace) then no other applies or plans can execute on that same combination.
2. No further changes to the (repository, directory, workspace) may be applied until the change is merged into the default branch or explicitly unlocked.
3. If an apply fails, no changes may be applied to the (repository, directory, workspace) outside of the pull request that failed.

These guarantees are desirable because:

1. They ensure updates are only applied serially.
2. Further changes are not possible until the applied change has made its way into the default branch, ensuring it’s visible in all further plans and applies.
3. On failure, the change is executed until it succeeds.

A locked repository is one where a pull request has some of its plans applied and is either not merged and/or not all plans are applied. There can only be one PR in a repository that contains a lock. A repository is unlocked by either:

1. The pull request is merged and all changed directories are applied.
2. A user explicitly issues a terrateam unlock comment on the pull request.

Terraform is also flexible in that the locking behavior can be modified. See the lock_policy configuration in the workflows documentation.

Hierarchy of Repositories

• The top is the repository
• A repository can contain zero or more directories
• A directory can contain one or more workspaces
• If no workspace is specified then each directory has a default workspace
• A workspace allows the contents of a directory to be used in different contexts
• A workspace maps to a Terraform workspace

Terrateam in Action

Creating a pull request with a Terraform code change

A pull request with a Terraform code change will automatically trigger a Plan operation.

Plan output

Terrateam will comment the Plan output to the pull request.

Request review

Request a review on your pull request and Plan output.

Trigger an Apply

The Apply operation will trigger on the approved pull request and terrateam apply comment.

Apply output and merge

Terrateam will comment the Apply output to the pull request.