bucket-events/

README

bucket-events

This module provisions infrastructure to listen to events from a GCS bucket and publish them to a broker.

flowchart LR
    G[Cloud Storage]
    G -- Pub/Sub --> T

    subgraph "regional network"
    T(Trampoline)
    I(Ingress)
    T -- emits --> I
    I -.-> E["..."]
    end

This uses Pub/Sub notifications for GCS and emits events of the following types:

GCS event type	CloudEvent type
OBJECT_FINALIZE	dev.chainguard.storage.object.finalize
OBJECT_METADATA_UPDATE	dev.chainguard.storage.object.metadata_update
OBJECT_DELETE	dev.chainguard.storage.object.delete
OBJECT_ARCHIVE	dev.chainguard.storage.object.archive

// Create a network with several regional subnets
module "networking" {
  source = "chainguard-dev/common/infra//modules/networking"

  name       = "my-networking"
  project_id = var.project_id
  regions    = [...] # Note: the bucket's location must be among these regions.
}

// Create the Broker abstraction.
module "cloudevent-broker" {
  source = "chainguard-dev/common/infra//modules/cloudevent-broker"

  name       = "my-broker"
  project_id = var.project_id
  regions    = module.networking.regional-networks
}

// Get the bucket.
data "google_storage_bucket" "bucket" {
  name = "my-bucket"
}

// Forward events to the broker.
module "bucket-events" {
  source = "./modules/bucket-events"

  project_id = local.project
  name       = "bucket-events"
  bucket     = data.google_storage_bucket.bucket.name
  regions    = module.networking.regional-networks
  ingress    = module.cloudevent-broker.ingress
}

Requirements

No requirements.

Providers

Name	Version
google	n/a
google-beta	n/a
random	n/a

Modules

Name	Source	Version
audit-delivery-serviceaccount	../audit-serviceaccount	n/a
authorize-delivery	../authorize-private-service	n/a
http	../dashboard/sections/http	n/a
layout	../dashboard/sections/layout	n/a
logs	../dashboard/sections/logs	n/a
resources	../dashboard/sections/resources	n/a
this	../regional-go-service	n/a
topic	../dashboard/sections/topic	n/a
trampoline-emits-events	../authorize-private-service	n/a
width	../dashboard/sections/width	n/a

Resources

Name	Type
google-beta_google_project_service_identity.pubsub	resource
google_monitoring_dashboard.dashboard	resource
google_pubsub_subscription.dead-letter-pull-sub	resource
google_pubsub_subscription.this	resource
google_pubsub_subscription_iam_binding.allow-pubsub-to-ack	resource
google_pubsub_topic.dead-letter	resource
google_pubsub_topic.internal	resource
google_pubsub_topic_iam_binding.allow-pubsub-to-send-to-dead-letter	resource
google_pubsub_topic_iam_binding.binding	resource
google_service_account.delivery	resource
google_service_account.service	resource
google_service_account_iam_binding.allow-pubsub-to-mint-tokens	resource
google_storage_notification.notification	resource
random_string.delivery-suffix	resource
random_string.service-suffix	resource
google_storage_bucket.bucket	data source
google_storage_project_service_account.gcs_account	data source

Inputs

Name	Description	Type	Default	Required
bucket	The name of the bucket to watch for events. The region where the bucket is located will be the region where the Pub/Sub topic and trampoline service will be created. The bucket must be in a region that is in the set of regions passed to the regions variable.	string	n/a	yes
enable_profiler	Enable cloud profiler.	bool	false	no
gcs_event_types	The types of GCS events to watch for (https://cloud.google.com/storage/docs/pubsub-notifications#payload).	list(string)	[ "OBJECT_FINALIZE", "OBJECT_METADATA_UPDATE", "OBJECT_DELETE", "OBJECT_ARCHIVE" ]	no
ingress	An object holding the name of the ingress service, which can be used to authorize callers to publish cloud events.	object({ name = string })	n/a	yes
max_delivery_attempts	The maximum number of delivery attempts for any event.	number	5	no
name	n/a	string	n/a	yes
notification_channels	List of notification channels to alert.	list(string)	n/a	yes
project_id	n/a	string	n/a	yes
regions	A map from region names to a network and subnetwork. The bucket must be in one of these regions.	map(object({ network = string subnet = string }))	n/a	yes

Outputs

No outputs.