Using LLMs to Generate Terraform Code - 2025 Update

Process

• A new chat prompt was opened with the o3 model selected.
• The prompt was entered into the model.

View Process Screenshots

Iteration #1

• The configurations were placed in their respective files as instructed: versions.tf, variables.tf, main.tf, outputs.tf.
• We've already hit a snag: the instructions ask the user to build and deploy their image before the infrastructure, which includes the ECR registry, is deployed. The model was prompted to correct this.
• Placeholder values were updated as instructed.

View Iteration #1 Screenshots

Iteration #2

• terraform init was run.
• Option "A" was chosen from what the model provided, terraform apply was run, targeting the ECR registry only.
• Terraform reported an error:

│ Error: Reference to undeclared resource
│
│   on main.tf line 106, in module "vpc":
│  106:   azs                  = slice(data.aws_availability_zones.available.names, 0, 2)
│
│ A data resource "aws_availability_zones" "available" has not been declared in the root module.

The error message was provided to the model, and the fix was applied.

View Iteration #2 Screenshots

Iteration #3

• terraform validate was used to confirm validity of the configuration, which succeeded.
• The terraform apply -target=aws_ecr_repository.app_repo command was attempted again, and was successful, but issued warnings around the use of targeted terraform apply.
• The build & push commands were run, however the following error was encountered when attempting to build the image:

ERROR: failed to build: invalid tag "\"************.dkr.ecr.us-east-1.amazonaws.com/hello-fargate-picked-sturgeon\":latest": invalid reference format

View Iteration #3 Screenshots

Iteration #4

• Option 2 of the fix was chosen, with a specific output added to outputs.tf
• terraform apply -refresh-only was run.
• The new output was set as a Bash variable with: REPO_URL=$(terraform output -raw ecr_repo_url)
• The build was run again: docker build -t ${REPO_URL}:latest ., and the image was built successfully.
• The following error was encountered when attempting to authenticate to ECR with: aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $REPO_URL:

Provided region_name '╷
│ Warning: No outputs found
│
│ The state file either has no outputs defined, or all the defined outputs are empty. Please define an output in your
│ configuration with the `output` keyword and run `terraform refresh` for it to become available. If you are using
│ interpolation, please verify the interpolated value is not empty. You can use the `terraform console` command to assist.
╵' doesn't match a supported format.
Error: Cannot perform an interactive login from a non TTY device

• The model was provided the error, and the fixes were applied.

View Iteration #4 Screenshots

Iteration #5

• aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $REPO_URL was run successfully.
• docker push ${REPO_URL}:latest was run, and the image was pushed successfully.
• terraform apply was run, and completed successfully.
• The app did not run successfully, with a 504 error being generated when viewing the frontend.
• Upon reviewing the task logs, it was discovered that the DynamoDB table name was not being injected into the container runtime.
• The model was provided the error, and the fix was applied.

raise ValueError("Please set the DYNAMODB_TABLE_NAME environment variable")

View Iteration #5 Screenshots

Iteration #6

• terraform apply was run.
• The task successfully started, however when visiting the ALB DNS URL, a Boto3 exception was thrown.
• The error was provided, and the fix was applied. NOTE: although the model recommends making changes to the application code, in order to maintain as much similarity to the previous test examples, we are implementing option B so that the application differs as little as possible from the previous test runs.

View Iteration #6 Screenshots

Iteration #7

• terraform apply was run successfully.
• The endpoint returned "Hello World" successfully.
• Timestamped entries were confirmed in the DynamoDB Table.

Notes

• Compared to the previous generation model, o3 provided much clearer instructions around file structure and where to place configuration blocks. It even made use of a publicly available module.

Issues

• The model did not identify the need to first bootstrap the ECR registry before docker images could be pushed and the remaining infrastructure deployed.
• A data source for availability zones was interpolated before it was created.
• The docker build command required changes.
• The command to authenticate to the ECR registry required new outputs to be defined in outputs.tf
• The environment variable name for the DynamoDB table did not match the application code*
• The primary key name for the DynamoDB table did not match what was defined in the application code*

Results

We got a working application! Although it required several iterations, most of the issues stemmed from either the initial bootstrapping and Docker commands, or configuration mismatches with the application code. I've marked the last two items with a "*", as those were not a direct result of model error, rather that the model was not supplied with the application code in its initial prompt and made assumptions.

My takeaway is that I am much more impressed with the overall IaC output and capabilities of the model. Compared to the previous OpenAI model, this configuration made much better use of value and variable interpolation, utilized public registry modules, and broke down the configuration into separate .tf files.

Process

• A new chat prompt was opened with the Claude Sonnet 4 model selected.
• The prompt was entered into the model.

View Process Screenshots

Iteration #1

• The model provided explicit instructions for set up, including a deploy.sh script, which was run after the initial directory was set up and files copied over.
• The deploy script was successfully run, and the app was verified as fully functional.

Notes

• Compared to the previous generation Claude model, this output had a much more perceptible focus on user experience; it provided a fully-featured deployment script that addressed the need for bootstrapping the ECR registry, as well as providing the outputs to the user.

Issues

• The Terraform configuration was generated entirely within a single file: main.tf. At minimum, most Terraform configurations start with separate files for the resources, variables, outputs, as well as provider and backend configuration.

Results

Very impressive. The model satisfied the requirements of the prompt in a single shot. The deployment script was a nice touch, and provided easy-to-read outputs, and did a good job of properly bootstrapping the resources to handle building and pushing a Docker image.

Process

• A new chat prompt was opened with the Le Chat Pro model selected.
• The prompt was entered into the model.

View Process Screenshots

Iteration #1

• The model provided some basic instructions for setup, including, creating and pushing a Docker image.
• The model did not correctly interpret the need to have created the ECR registry before pushing the image.
• The model was prompted with this information, and the fixes were applied.

View Iteration #1 Screenshots

Iteration #2

• terraform init, terraform plan, and terraform apply were run successfully.
• The Docker image was built and pushed to the ECR registry.
• aws ecs update-service --cluster flask-app-cluster --service flask-app-service --force-new-deployment was run successfully.
• When visiting the endpoint, a 504 error was generated. The model was prompted with this information.

View Iteration #2 Screenshots

Iteration #3

• Since the model provided only generic troubleshooting advice, it was prompted to fix the issue again. At this point, I noticed the model required manually enabling "Think" mode, which was done.

View Iteration #3 Screenshots

Iteration #4

• The new Terraform configuration was copied, and the terraform init, terraform plan, and terraform apply commands were run successfully.
• When visiting the endpoint, an exception was thrown.
• The model was prompted with the exception information and asked to fix it.

View Iteration #4 Screenshots

Iteration #5

• The model again asked the user to perform generic troubleshooting steps, and was prompted to fix the issue.

View Iteration #5 Screenshots

Iteration #6

• The new Terraform configuration was copied over, and terraform init, plan, apply commands were run.
• An error was received:

│ Error: creating IAM Role (ecsTaskExecutionRole): EntityAlreadyExists: Role with name ecsTaskExecutionRole already exists.
│       status code: 409, request id: 69650f31-7e87-49d5-bddf-10926e57010c
│
│   with aws_iam_role.ecs_task_execution_role,
│   on main.tf line 52, in resource "aws_iam_role" "ecs_task_execution_role":
│   52: resource "aws_iam_role" "ecs_task_execution_role" {
│

• On further investigation, it was also discovered the Terraform configuration contained the same error that had plagued the previous generation models: it did not create separate execution and task roles.
• The model was prompted with this information. (NOTE: after generating a new config and instructions, the model generated a large amount of repeating text.)

View Iteration #6 Screenshots

Iteration #7

• The new Terraform configuration was copied over, and terraform init, plan, apply commands were run successfully.
• aws ecs update-service --cluster flask-app-cluster --service flask-app-service --force-new-deployment was run to ensure the latest task definition was being used.
• The endpoint successfully returned "Hello World", with entries written to the DynamoDB table.

Notes

• Unfortunately, this felt like I was still interacting with a previous-generation LLM model. It frequently asked the user to perform generic troubleshooting steps rather than fixing its own work, and made several errors.

Issues

• The model frequently defaulted to asking the user to perform generic troubleshooting steps.
• The model did not correctly interpret that the ECR registry would need to be created before a Docker image could be pushed.
• The need for separate task and execution roles was not properly addressed until specifically called out by the user.
• A configuration was generated that would not even complete a terraform apply without failure.
• A single Terraform file was generated, rather than multiple files.
• The model asked the user to perform several manual commands, rather than attempting to fulfill the requirement that the user should have only minimal intervention.

Results

Even with "Thinking" mode turned on, the Le Chat model very much gave the impression that I was still interacting with a previous generation of LLM. The issues and rough edges I encountered were all very typical for the models that were tested over a year ago. Eventually we did converge on a working application, but it felt like a lot of work to arrive at that point.

Process

• A new chat prompt was opened with the Gemini 2.5 Pro model selected.
• The prompt was entered into the model.

View Process Screenshots

Iteration #1

• The Terraform configurations were copied to their respective files.
• terraform init and terraform apply -auto-approve were run per model instructions.
• The commands to build and push the Docker image were run successfully.
• The outputs were added to the outputs.tf file.
• Per instructions, terraform apply -auto-approve was run successfully.
• The ecs service was updated via aws ecs update-service --cluster $(terraform output -raw ecs_cluster_name) --service $(terraform output -raw ecs_service_name) --force-new-deployment --region
• The app successfully returned "Hello World", and the DynamoDB table was populated with items.

Notes

• This is the first time testing the Gemini 2.5 model.

Issues

• This is more of a nit, but there was more manual work for the user compared to the Claude Sonnet 4 model, which bundled all commands in a deployment script.

Results

The Gemini 2.5 model jumps out of the gate with a great showing, managing to produce a working configuration in one pass. It also produced the cleanest output, breaking down Terraform configuration into multiple files that were easy to access via the chat UI. The Terraform configuration also made use of variables and interpolation as well; very impressive.