github-events/

README

github-events

This module provisions infrastructure to listen to webhook events from GitHub and publish them to a broker.

flowchart LR
    G[GitHub.com]
    G -- webhook --> T

    subgraph "regional network"
    T(Trampoline)
    I(Ingress)
    T -- emits --> I
    I -.-> E["..."]
    end

More information on GitHub webhooks:

• https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries
• https://docs.github.com/en/webhooks/webhook-events-and-payloads

Events are published as they are received from GitHub, and are not transformed in any way. The events are published to a broker, which can be used to fan out the events to other services, or filter based on their type.

You can use this with cloudevent-recorder to record GitHub events to a BigQuery table.

CloudEvent types are derived from the GitHub event type, and are prefixed with dev.chainguard.github. For example, the push event is published as dev.chainguard.github.push.

// Create a network with several regional subnets
module "networking" {
  source = "chainguard-dev/common/infra//modules/networking"

  name       = "my-networking"
  project_id = var.project_id
  regions    = [...]
}

// Create the Broker abstraction.
module "cloudevent-broker" {
  source = "chainguard-dev/common/infra//modules/cloudevent-broker"

  name       = "my-broker"
  project_id = var.project_id
  regions    = module.networking.regional-networks
}

// Forward events to the broker.
module "github-events" {
  source = "./modules/github-events"

  project_id = var.project_id
  name       = "github-events"
  regions    = module.networking.regional-networks
  ingress    = module.cloudevent-broker.ingress

  // Which user is allowed to populate webhook secret values.
  secret_version_adder = "user:you@company.biz"
}

After applying this, generate a random secret value and add it to the GitHub webhook config, and populate the secret version in the GCP Secret Manager.

Using with serverless-gclb

To expose the service to the internet for production, you should use serverless-gclb to create a load-balanced public endpoint. This is the endpoint where GitHub will be configured to send webhook requests.

data "google_dns_managed_zone" "top-level-zone" { name = "your-top-level-zone" }

module "serverless-gclb" {
  source = "chainguard-dev/common/infra//modules/serverless-gclb"

  name       = "github-events"
  project_id = var.project_id
  dns_zone   = data.google_dns_managed_zone.top-level-zone.name

  // Regions are all of the places that we have backends deployed.
  // Regions must be removed from serving before they are torn down.
  regions         = keys(module.networking.regional-networks)
  serving_regions = keys(module.networking.regional-networks)

  public-services = {
    // Matches github-events module name.
    "github.yourdomain.com" = { name = "github-events" }
  }
}

Using with INGRESS_TRAFFIC_ALL

During development you may want to expose the service directly to the internet, without using a load balancer. This is useful for testing and development, but is not recommended in production.

module "github-events" {
  source = "./modules/github-events"

  project_id = var.project_id
  name       = "github-events"
  regions    = module.networking.regional-networks
  ingress    = module.cloudevent-broker.ingress

  service-ingress = "INGRESS_TRAFFIC_ALL" // Expose the service to the internet.
}

The public-urls output will be populated with the .run.app URL for each regional service, which can be used to configure the GitHub webhook for testing.

Using with cloudevent-recorder

The event payloads produced by this module are the full GitHub webhook payloads, and are not transformed in any way. If you want to record these events using cloudevent-recorder, you must set ignore_unknown_fields, since event payloads will not match the schema.

The schemas that describe which fields get recorded are defined in ./schemas/event_types.go, and the BQ schemas are generated using ./cmd/schemagen. To add fields or new types, modify the event_types.go file and run go generate ./....

Requirements

No requirements.

Providers

Name	Version
google	n/a
random	n/a

Modules

Name	Source	Version
dashboard	../dashboard	n/a
http	../dashboard/sections/http	n/a
layout	../dashboard/sections/layout	n/a
logs	../dashboard/sections/logs	n/a
resources	../dashboard/sections/resources	n/a
this	../regional-go-service	n/a
trampoline-emits-events	../authorize-private-service	n/a
webhook-secret	../secret	n/a
width	../dashboard/sections/width	n/a

Resources

Name	Type
google_service_account.service	resource
random_string.service-suffix	resource
google_cloud_run_v2_service.this	data source

Inputs

Name	Description	Type	Default	Required
additional_webhook_secrets	Additional secrets to be used by the service. - key: Local identifier for the secret. This will be prefixed with WEBHOOK_SECRET_ in the service's environment vars. - secret: The name of the secret in Cloud Secret Manager. Format: {secretName} if the secret is in the same project. projects/{project}/secrets/{secretName} if the secret is in a different project. - version: The version of the secret to use. Can be a number or 'latest'. See https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_v2_service#nested_env for related documentation.	map(object({ secret = string version = string }))	{}	no
deletion_protection	Whether to enable delete protection for the service.	bool	true	no
enable_profiler	Enable cloud profiler.	bool	false	no
github_organizations	csv string of GitHub organizations to allow.	string	""	no
ingress	An object holding the name of the ingress service, which can be used to authorize callers to publish cloud events.	object({ name = string })	n/a	yes
max_delivery_attempts	The maximum number of delivery attempts for any event.	number	5	no
name	n/a	string	n/a	yes
notification_channels	List of notification channels to alert.	list(string)	n/a	yes
project_id	n/a	string	n/a	yes
regions	A map from region names to a network and subnetwork. The bucket must be in one of these regions.	map(object({ network = string subnet = string }))	n/a	yes
requested_only_webhook_id	If set, the csv IDs of the webhooks that should only receive check requested events.	string	""	no
require_squad	Whether to require squad variable to be specified	bool	false	no
secret_version_adder	The user allowed to populate new webhook secret versions.	string	n/a	yes
service-ingress	Which type of ingress traffic to accept for the service (see regional-go-service). Valid values are: - INGRESS_TRAFFIC_ALL accepts all traffic, enabling the public .run.app URL for the service - INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER accepts traffic only from a load balancer	string	"INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER"	no
squad	squad label to apply to the service.	string	""	no
webhook_id	If set, the csv IDs of the webhooks that the trampoline should listen to.	string	""	no

Outputs

Name	Description
public-urls	Map of region to public URL for the service, if service-ingress is INGRESS_TRAFFIC_ALL.
recorder-schemas	READ THIS BEFORE YOU EDIT!!! These schemas are used to generate bigquery table names used by the recorder. If you are adding a schema you're fine to proceed. If you are changing the name of a schema, or removing a schema, terraform will try to delete the old schema. The recorders have a parameter deletion_protection enabled by default so terraform will fail to delete the schema. The proper process for deleting or modifying a schema is in this playbook https://eng.inky.wtf/docs/infra/playbooks/schema-names/