How Terrateam Works
Terrateam is a GitHub application that translates GitHub events into Terraform executions. There are two major components of the Terrateam service:
- The backend which receives GitHub events and makes decisions using the event payload
- The Terrateam runner which executes the jobs that the backend creates
Execution Steps
- A user performs an operation, such as opening a pull request, closing one, or commenting on one. This results in an event being sent to the Terrateam backend.
- The Terrateam backend evaluates the event to determine if the GitHub Action should be run. If there are changes that require a Terraform run, the backend triggers a GitHub Action workflow dispatch.
- The Terrateam GitHub Action executes the required work, generally running a Terraform operation in each directory that has changed. Once complete, it publishes the results back to the Terrateam backend.
- Finally, the Terrateam backend publishes any results back to the pull request in the form of a comment as well as updating any status checks.
Operations
There are two Terraform operations that can be executed with Terrateam:
- Plan
- Apply
Terrateam references these operations internally as the following:
- Plan
- Autoplan
- Apply
- Autoapply
The distinction between a plan
and apply
are initiated manually
by a customer and the auto
versions are initiated indirectly due
to a customer action such as creating a change, pushing an update to
a change, or merging a change.
Plan
Plans have the following semantics (per repository):
- Multiple
plan
operations can be executing concurrently. - A
plan
cannot be executing if anapply
is executing. - A
plan
can be repeatedly executed.
Apply
Applies have the following semantics (per repository):
- An
apply
has higher priority than aplan
. - An
apply
is mutually exclusive with all other operations. - All changes in the
apply
must have a plan on the change id. - The most recent
plan
for all changes that will be applied must be successful. - The change must either have no other applies, or be explicitly unlocked, or any applied changes must be merged into the default branch.
Event Evaluation
To evaluate a GitHub event, one of the following must be true:
- Pull request open
- Pull request sync
- Pull request close
- Issue comment
- Comment is
terrateam plan ...
- Comment is
terrateam apply ...
- Comment is
If the event corresponds to a possible work manifest, the pull request is then evaluated by the Terrateam backend.
Executing a GitHub Actions Workflow
Receiving Events
The events the Terrateam backend operates on:
- Installation event:
- Create
- Suspend
- Resume
- Uninstall
- Update to pull request:
- Create a pull request
- Synchronize a pull request (updating it)
- Closing a pull request (either by merge or just closing)
- Commenting:
terrateam plan
- Execute a plan.terrateam apply
- Execute an apply.
In the case of event types (2) and (3), we will construct a work manifest which may be executed. The output of this step is what pull request to operate on and if it is a manual or automatic operation.
Evaluating a Pull Request and Enqueuing
Terrateam will fetch the current state of the PR and collect the following information:
- The state of the PR:
open
,closed
,merged
. - The destination branch.
- If the destination branch is the
default_branch
. - The
base_sha
and thesha
of the PR. - The
(dir, workspace)
list of changed files between the sha’s. - The
(dir, workspace)
list of changed files when applyingrun_type
andtag_query
.
Executing the Work Manifest
After any change in state, the Terrateam backend initiates the execution of any work manifests by performing the following steps:
- Select all work manifests that are
queued
- For each work manifest, group by repository and rank based on priority and
creation date.
autoapply
andapply
have higher priority thanautoplan
andplan
.- Creation date is ordered oldest to newest.
- Sort all work manifests by creation date and take the top ranked work manifest for each repository
- Skip any rows that are being locked by a concurrent update.
- Limit results to the first row.
- Set state of work manifest to
running
.
Locking
Terrateam ensures that changes are safely applied. The backend guarantees the following:
- If an apply is executing on a (
repository
,directory
,workspace
) then no other applies or plans can execute on that same combination. - No further changes to the (
repository
,directory
,workspace
) may be applied until the change is merged into the default branch or explicitly unlocked. - If an apply fails, no changes may be applied to the (
repository
,directory
,workspace
) outside of the pull request that failed.
These guarantees are desirable because:
- They ensure updates are only applied serially.
- Further changes are not possible until the applied change has made its way into the default branch, ensuring it’s visible in all further plans and applies.
- On failure, the change is executed until it succeeds.
A locked repository is one where a pull request has some of its plans applied and is either not merged and/or not all plans are applied. There can only be one PR in a repository that contains a lock. A repository is unlocked by either:
- The pull request is merged and all changed directories are applied.
- A user explicitly issues a
terrateam unlock
comment on the pull request.
Terraform is also flexible in that the locking behavior can be modified. See
the lock_policy
configuration in the workflows
documentation.
Hierarchy of Repositories
- The top is the repository
- A repository can contain zero or more directories
- A directory can contain one or more workspaces
- If no workspace is specified then each directory has a default workspace
- A workspace allows the contents of a directory to be used in different contexts
- A workspace maps to a Terraform workspace
Terrateam in Action
Creating a pull request with a Terraform code change
A pull request with a Terraform code change will automatically trigger a Plan operation.
Plan output
Terrateam will comment the Plan output to the pull request.
Request review
Request a review on your pull request and Plan output.
Trigger an Apply
The Apply operation will trigger on the approved pull request and terrateam apply
comment.
Apply output and merge
Terrateam will comment the Apply output to the pull request.