I recently came across an r/terraform post about a GitHub project called terraform-registry-proxy.
This inspired me to create a similar project strictly using Nginx.
(Thanks u/jasonwbarnett!)
What is a Terraform Registry Cache?
By default, Terraform will pull down provider artifacts from https://releases.hashicorp.com. For the typical user this behavior is acceptable. For environments that are air gapped or require a local caching server, we can do better.
How does this work exactly?
Nginx will proxy requests for the following endpoints:
Response bodies are rewritten to update where provider artifacts are served from. Terraform will use the rewritten URL to fetch the artifact from the registry cache (Nginx).
Nginx will serve back a response from its local cache if possible and fallback to the official Hashicorp servers.
Requirements
- An Nginx server with SSL configured. The Terraform CLI must be able to trust the certificate.
- Two subdomains:
- terraform-registry.example.com
- terraform-releases.example.com
Nginx Configuration
This is an example nginx.conf. Modify as you see fit.
user nginx;worker_processes auto;error_log /var/log/nginx/error.log notice;pid /var/run/nginx.pid;
events { worker_connections 1024;}
http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; gzip on;
proxy_headers_hash_bucket_size 128; proxy_cache_path /tmp/terraform-cache levels=1:2 keys_zone=terraform_cache:10m max_size=10g inactive=7d use_temp_path=off;
# Custom log format so we can validate cache HITS and MISSES log_format custom_cache_log '[$time_local] [Cache:$upstream_cache_status] [$host] [Remote_Addr: $remote_addr] - $remote_user - $server_name to: $upstream_addr: "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" ';
# terraform-registry.example.com server { listen 443 ssl; server_name terraform-registry.example.com; resolver 8.8.8.8 ipv6=off; access_log /var/log/nginx/terraform-registry-cache.log custom_cache_log;
# SSL ssl_certificate "/etc/letsencrypt/live/terraform-registry.example.com/fullchain.pem"; ssl_certificate_key "/etc/letsencrypt/live/terraform-registry.example.com/privkey.pem"; ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DH+3DES:!ADH:!AECDH:!MD5; ssl_session_cache shared:SSL:1m; ssl_session_timeout 10m; ssl_prefer_server_ciphers on;
location / { # Pull from the local cache then https://registry.terraform.io/ proxy_cache terraform_cache; proxy_cache_use_stale error timeout; proxy_pass https://registry.terraform.io/; proxy_set_header Accept-Encoding ""; proxy_set_header X-Forwarded-For $remote_addr; tcp_nodelay on; # Rewrite releases.hashicorp.com payloads so the Terraform CLI pulls artifacts from terraform-releases.example.com sub_filter_types application/json; sub_filter 'releases.hashicorp.com' 'terraform-releases.example.com'; } }
# terraform-releases.example.com server { listen 443 ssl; server_name terraform-releases.example.com; resolver 8.8.8.8 ipv6=off; access_log /var/log/nginx/terraform-releases-cache.log custom_cache_log;
# SSL ssl_certificate "/etc/letsencrypt/live/terraform-releases.example.com/fullchain.pem"; ssl_certificate_key "/etc/letsencrypt/live/terraform-releases.example.com/privkey.pem"; ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DH+3DES:!ADH:!AECDH:!MD5; ssl_session_cache shared:SSL:1m; ssl_session_timeout 10m; ssl_prefer_server_ciphers on;
location / { # Pull from the local cache then https://releases.hashicorp.com/ proxy_cache terraform_cache; proxy_cache_use_stale error timeout; proxy_pass https://releases.hashicorp.com/; proxy_set_header Accept-Encoding ""; proxy_set_header X-Forwarded-For $remote_addr; tcp_nodelay on; } }}Terraform Configuration
Inside your main.tf, set the global source address against the provider:
terraform { required_providers { aws = { source = "terraform-registry.example.com/hashicorp/aws" } }}Example Usage
Inside the directory with your main.tf changes:
$ terraform init$ terraform initjosh@ringo:~/$ tail -f /var/log/nginx/terraform-registry-cache.log[08/Apr/2022:12:44:39 +0000] [Cache:MISS] [registry.terrateam.io] [Remote_Addr: 213.93.7.141] - - - registry.terrateam.io to: 151.101.86.49:443: "GET /.well-known/terraform.json HTTP/1.1" 200 73 "-" "HashiCorp Terraform/1.1.7 (+https://www.terraform.io)"[08/Apr/2022:12:44:39 +0000] [Cache:MISS] [registry.terrateam.io] [Remote_Addr: 213.93.7.141] - - - registry.terrateam.io to: 151.101.86.49:443: "GET /v1/providers/hashicorp/aws/versions HTTP/1.1" 200 96922 "-" "Terraform/1.1.7"[08/Apr/2022:12:44:39 +0000] [Cache:MISS] [registry.terrateam.io] [Remote_Addr: 213.93.7.141] - - - registry.terrateam.io to: 151.101.86.49:443: "GET /v1/providers/hashicorp/aws/4.9.0/download/linux/amd64 HTTP/1.1" 200 8598 "-" "Terraform/1.1.7"[08/Apr/2022:12:44:55 +0000] [Cache:HIT] [registry.terrateam.io] [Remote_Addr: 213.93.7.141] - - - registry.terrateam.io to: -: "GET /.well-known/terraform.json HTTP/1.1" 200 73 "-" "HashiCorp Terraform/1.1.7 (+https://www.terraform.io)"[08/Apr/2022:12:44:55 +0000] [Cache:HIT] [registry.terrateam.io] [Remote_Addr: 213.93.7.141] - - - registry.terrateam.io to: -: "GET /v1/providers/hashicorp/aws/versions HTTP/1.1" 200 96882 "-" "Terraform/1.1.7"[08/Apr/2022:12:44:55 +0000] [Cache:HIT] [registry.terrateam.io] [Remote_Addr: 213.93.7.141] - - - registry.terrateam.io to: -: "GET /v1/providers/hashicorp/aws/4.9.0/download/linux/amd64 HTTP/1.1" 200 8591 "-" "Terraform/1.1.7"josh@ringo:~/terraform-registry-cache$josh@ringo:~/$ tail -f /var/log/nginx/terraform-releases-cache.log[08/Apr/2022:12:44:42 +0000] [Cache:MISS] [releases.terrateam.io] [Remote_Addr: 213.93.7.141] - - - releases.terrateam.io to: 151.101.85.183:443: "GET /terraform-provider-aws/4.9.0/terraform-provider-aws_4.9.0_linux_amd64.zip HTTP/1.1" 200 53259564 "-" "Terraform/1.1.7"[08/Apr/2022:12:44:58 +0000] [Cache:HIT] [releases.terrateam.io] [Remote_Addr: 213.93.7.141] - - - releases.terrateam.io to: -: "GET /terraform-provider-aws/4.9.0/terraform-provider-aws_4.9.0_linux_amd64.zip HTTP/1.1" 200 53259564 "-" "Terraform/1.1.7"josh@ringo:~/terraform-registry-cache$You’ll notice I ran the init command two times.
This is to show that the first time Nginx served the request produced a Cache:MISS . The second time Nginx was able to pull from the local cache resulting in a Cache:HIT .
Conclusion
This Nginx setup could easily be integrated against environments that don’t have a direct connection to the Internet or company policy requiring the use of proxy servers.
Sign up for Terrateam here!