Skip to content

Instantly share code, notes, and snippets.

@Razzendah

Razzendah/README.md

Forked from markmssd/README.md
Created May 30, 2024 18:51
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save Razzendah/67c34c1d2068bae26441956737b27743 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save Razzendah/67c34c1d2068bae26441956737b27743 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
README.md

Google Cloud CDN with private GCS bucket

How to authenticate Google Cloud CDN to a private GCS bucket

In order to authenticate to a private GCS bucket, we need to connect the Cloud CDN load balancer through a Backend Service, instead of a Backend Bucket. See Configure private origin authentication for more details.

As of April 2024, the Terraform compute_backend_service resource does not support setting security_settings.aws_v4_authentication, even though it should be.

Until this is supported, we have to set it manually via gcloud. Below are the steps, to be run AFTER the Terraform plan has been applied:

  1. Start by updating your gcloud cli, as older versions do not support setting securitySettings.awsV4Authentication:

    gcloud components update

  2. Export the current backend service resource:

    gcloud compute backend-services export cdn-backend-service --destination cdn-backend-service.yaml --global

  3. Prepend the securitySettings block at the end of the file:

    ...
selfLink: ...
sessionAffinity: ...
timeoutSec: ...
securitySettings:
  awsV4Authentication:
    accessKeyId: <access_key_id>
    accessKey: <access_key_secret>
    originRegion: us-central1

Refer to Manage HMAC keys for service accounts to create your HMAC access key.

  1. Save the new configuration: 
    gcloud compute backend-services import cdn-backend-service --source cdn-backend-service.yaml --global
rm cdn-backend-service.yaml # delete as it contains secrets!

That's it!

Kudos to Navya Dwarakanath for her great article https://medium.com/@thetechbytes/private-gcs-bucket-access-through-google-cloud-cdn-430d940ebad9.

Warning

It is worth noting that, since we set up securitySettings.awsV4Authentication via gcloud, any future updates to the Terraform compute_backend_service resource will fail. There is already an issue opened in this regard: hashicorp/terraform-provider-google#16897.

To get unblocked, we need to follow the same steps as above to remove the securitySettings block, apply the Terraform plan, then follow the steps again to re-add the securitySettings.awsV4Authentication. There will be CDN downtime during this operation. This is only an issue because we use Terraform.

Raw
cdn.tf
# Private GCS bucket
resource "google_storage_bucket" "cdn_bucket" {
name = "gcdn-unreal-cloud-test"
location = "US"
storage_class = "MULTI_REGIONAL"
force_destroy = false
uniform_bucket_level_access = true
public_access_prevention = "enforced"
versioning {
enabled = false
}
}
# Load Balancer
resource "google_compute_global_address" "cdn_ip" {
name = "cdn-ip"
ip_version = "IPV4"
address_type = "EXTERNAL"
}
# Add the IP as an A Record to your domain registrar, and wait for certificate to be provisioned
resource "google_compute_managed_ssl_certificate" "cdn_certificate" {
name = "cdn-certificate"
managed {
domains = [
"cdn.yourcdndomain.com",
]
}
}
resource "google_compute_global_network_endpoint_group" "cdn_network_endpoint_group" {
name = "cdn-network-endpoint-group"
network_endpoint_type = "INTERNET_FQDN_PORT"
default_port = 443
}
resource "google_compute_global_network_endpoint" "cdn_network_endpoint" {
global_network_endpoint_group = google_compute_global_network_endpoint_group.cdn_network_endpoint_group.id
fqdn = "${google_storage_bucket.cdn_bucket.name}.storage.googleapis.com"
port = google_compute_global_network_endpoint_group.cdn_network_endpoint_group.default_port
}
# Updating this resource after setting `securitySettings.awsV4Authentication` might fail: https://github.com/hashicorp/terraform-provider-google/issues/16897
resource "google_compute_backend_service" "cdn_backend_service" {
name = "cdn-backend-service"
description = "Backend service for Google Cloud CDN"
load_balancing_scheme = "EXTERNAL_MANAGED"
compression_mode = "AUTOMATIC"
protocol = "HTTPS"
enable_cdn = true
backend {
group = google_compute_global_network_endpoint_group.cdn_network_endpoint_group.id
}
custom_request_headers = [
"Host:${google_storage_bucket.cdn_bucket.name}.storage.googleapis.com",
"Cookie:", # unset cookies to avoid HMAC authentication failure
]
custom_response_headers = [
"X-Cache-Hit:{cdn_cache_status}"
]
cdn_policy {
cache_mode = "FORCE_CACHE_ALL"
negative_caching = false
cache_key_policy {
include_host = true
include_query_string = true
}
bypass_cache_on_request_headers {
header_name = "X-Bypass-Cache"
}
}
lifecycle {
ignore_changes = [
# securitySettings gets set via gcloud, thus prevent Terraform from deleting it
security_settings,
]
}
}
resource "google_compute_url_map" "cdn_lb" {
name = "cdn-lb"
description = "Load Balancer to redirect requests to bucket backend"
default_service = google_compute_backend_service.cdn_backend_service.id
}
resource "google_compute_target_https_proxy" "cdn_https_proxy" {
name = "cdn-https-proxy"
url_map = google_compute_url_map.cdn_lb.self_link
ssl_certificates = [google_compute_managed_ssl_certificate.cdn_certificate.self_link]
}
resource "google_compute_global_forwarding_rule" "cdn_https_forwarding_rule" {
name = "cdn-https-forwarding-rule"
target = google_compute_target_https_proxy.cdn_https_proxy.self_link
ip_address = google_compute_global_address.cdn_ip.address
load_balancing_scheme = "EXTERNAL_MANAGED"
port_range = "443"
}
# Partial load balancer for https redirects
resource "google_compute_url_map" "cdn_lb_https_redirect" {
name = "cdn-lb-https-redirect"
description = "Partial Load Balancer for HTTPS Redirects"
default_url_redirect {
https_redirect = true
strip_query = false
}
}
resource "google_compute_target_http_proxy" "cdn_http_proxy" {
name = "cdn-http-proxy"
url_map = google_compute_url_map.cdn_lb_https_redirect.id
}
resource "google_compute_global_forwarding_rule" "cdn_http_forwarding_rule" {
name = "cdn-http-forwarding-rule"
target = google_compute_target_http_proxy.cdn_http_proxy.id
ip_address = google_compute_global_address.cdn_ip.id
load_balancing_scheme = "EXTERNAL_MANAGED"
port_range = "80"
}
# Service Account
resource "google_service_account" "cdn_bucket_service_account" {
account_id = "cdn-service-account"
display_name = "CDN Service Account"
}
resource "google_storage_bucket_iam_member" "google_cdn_bucket_object_admin" {
bucket = google_storage_bucket.cdn_bucket.name
role = "roles/storage.legacyObjectReader"
member = "serviceAccount:${google_service_account.cdn_bucket_service_account.email}"
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment