github-events/

README

github-events

This module provisions infrastructure to listen to webhook events from GitHub and publish them to a broker.

flowchart LR
    G[GitHub.com]
    G -- webhook --> T

    subgraph "regional network"
    T(Trampoline)
    I(Ingress)
    T -- emits --> I
    I -.-> E["..."]
    end

More information on GitHub webhooks:

• https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries
• https://docs.github.com/en/webhooks/webhook-events-and-payloads

Events are published as they are received from GitHub, and are not transformed in any way. The events are published to a broker, which can be used to fan out the events to other services, or filter based on their type.

You can use this with cloudevent-recorder to record GitHub events to a BigQuery table.

CloudEvent types are derived from the GitHub event type, and are prefixed with dev.chainguard.github. For example, the push event is published as dev.chainguard.github.push.

// Create a network with several regional subnets
module "networking" {
  source = "chainguard-dev/common/infra//modules/networking"

  name       = "my-networking"
  project_id = var.project_id
  regions    = [...]
}

// Create the Broker abstraction.
module "cloudevent-broker" {
  source = "chainguard-dev/common/infra//modules/cloudevent-broker"

  name       = "my-broker"
  project_id = var.project_id
  regions    = module.networking.regional-networks
}

// Forward events to the broker.
module "github-events" {
  source = "./modules/github-events"

  project_id = var.project_id
  name       = "github-events"
  regions    = module.networking.regional-networks
  ingress    = module.cloudevent-broker.ingress

  // Which user is allowed to populate webhook secret values.
  secret_version_adder = "user:you@company.biz"
}

After applying this, generate a random secret value and add it to the GitHub webhook config, and populate the secret version in the GCP Secret Manager.

Using with serverless-gclb

To expose the service to the internet for production, you should use serverless-gclb to create a load-balanced public endpoint. This is the endpoint where GitHub will be configured to send webhook requests.

data "google_dns_managed_zone" "top-level-zone" { name = "your-top-level-zone" }

module "serverless-gclb" {
  source = "chainguard-dev/common/infra//modules/serverless-gclb"

  name       = "github-events"
  project_id = var.project_id
  dns_zone   = data.google_dns_managed_zone.top-level-zone.name

  // Regions are all of the places that we have backends deployed.
  // Regions must be removed from serving before they are torn down.
  regions         = keys(module.networking.regional-networks)
  serving_regions = keys(module.networking.regional-networks)

  public-services = {
    // Matches github-events module name.
    "github.yourdomain.com" = { name = "github-events" }
  }
}

Using with INGRESS_TRAFFIC_ALL

During development you may want to expose the service directly to the internet, without using a load balancer. This is useful for testing and development, but is not recommended in production.

module "github-events" {
  source = "./modules/github-events"

  project_id = var.project_id
  name       = "github-events"
  regions    = module.networking.regional-networks
  ingress    = module.cloudevent-broker.ingress

  service-ingress = "INGRESS_TRAFFIC_ALL" // Expose the service to the internet.
}

The public-urls output will be populated with the .run.app URL for each regional service, which can be used to configure the GitHub webhook for testing.

Using with cloudevent-recorder

The event payloads produced by this module are the full GitHub webhook payloads, and are not transformed in any way. If you want to record these events using cloudevent-recorder, you must set ignore_unknown_fields, since event payloads will not match the schema.

The schemas that describe which fields get recorded are defined in ./schemas/event_types.go, and the BQ schemas are generated using ./cmd/schemagen. To add fields or new types, modify the event_types.go file and run go generate ./....

Requirements

No requirements.

Providers

Name	Version
google	n/a
random	n/a

Modules

Name	Source	Version
dashboard	../dashboard	n/a
http	../dashboard/sections/http	n/a
layout	../dashboard/sections/layout	n/a
logs	../dashboard/sections/logs	n/a
resources	../dashboard/sections/resources	n/a
this	../regional-go-service	n/a
trampoline-emits-events	../authorize-private-service	n/a
webhook-secret	../secret	n/a
width	../dashboard/sections/width	n/a

Resources

Name	Type
google_service_account.service	resource
random_string.service-suffix	resource
google_cloud_run_v2_service.this	data source

Inputs

Name	Description	Type	Default	Required
deletion_protection	Whether to enable delete protection for the service.	bool	true	no
enable_profiler	Enable cloud profiler.	bool	false	no
ingress	An object holding the name of the ingress service, which can be used to authorize callers to publish cloud events.	object({ name = string })	n/a	yes
max_delivery_attempts	The maximum number of delivery attempts for any event.	number	5	no
name	n/a	string	n/a	yes
notification_channels	List of notification channels to alert.	list(string)	n/a	yes
project_id	n/a	string	n/a	yes
regions	A map from region names to a network and subnetwork. The bucket must be in one of these regions.	map(object({ network = string subnet = string }))	n/a	yes
require_squad	Whether to require squad variable to be specified	bool	false	no
secret_version_adder	The user allowed to populate new webhook secret versions.	string	n/a	yes
service-ingress	Which type of ingress traffic to accept for the service (see regional-go-service). Valid values are: - INGRESS_TRAFFIC_ALL accepts all traffic, enabling the public .run.app URL for the service - INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER accepts traffic only from a load balancer	string	"INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER"	no
squad	squad label to apply to the service.	string	""	no

Outputs

Name	Description
public-urls	Map of region to public URL for the service, if service-ingress is INGRESS_TRAFFIC_ALL.
recorder-schemas	READ THIS BEFORE YOU EDIT!!! These schemas are used to generate bigquery table names used by the recorder. If you are adding a schema you're fine to proceed. If you are changing the name of a schema, or removing a schema, terraform will try to delete the old schema. The recorders have a parameter deletion_protection enabled by default so terraform will fail to delete the schema. The proper process for deleting or modifying a schema is in this playbook https://eng.inky.wtf/docs/infra/playbooks/schema-names/