kubeci

package module
v0.12.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 28, 2023 License: Apache-2.0 Imports: 51 Imported by: 0

README

Github CI with K8S and Argo.

kube-ci builds on argo, and the github API, to provide CI/CD workflows. The design goal is to provide as much feedback and support directly via the github API, rather than through other interfaces.

What's here

The code is still work-in-progress but has seen extensive use for a few years now.

If a given commit has a workflow file, location in .kube-ci/ci.yaml, kube-ci:

  • Configuration is taken directly from the repository when various github events occur.
  • Creates a Check Run when a Check Suite is requested. (see Build Policy below)
  • Creates the a workflow based on the provided file.
  • Reports parsing errors of the user supplied workflow back the Github UI.
  • Feeds back the status of Workflow pods to the Github UI
  • Links to the argo-ui for the created workflow
  • Populates file:N:M: message from logs as Annotations in GitHub check runs.
  • You can include configurations which can be imported by comments on github PRs/issues
  • Optionally create a PVC, either per-repo or per-repo-branch, that can be used for cacheing between runs. (clean up of these is yet to be implemented)
  • Allows workflows runs to be manually trigger from a PR using /kube-ci commands.
  • Allows initial bootstrap of the workflows via templates and a /kube-ci command from issues and PRs.
  • Allows filtering responses to limit them to specific orgs and install IDs

CI Configuration

By default kube-ci will look for configuration in the configured CI Context directory (default: .kube-ci) of the repository being built. Winthin that directory kube-ci will check for the following files in order:

  • ci.star: If present this file is treated as a Starlark configuration file and executed. Please see STARLARK.md for further details

  • ci.yaml: If present this is read as a direct YAML workflow definition

(The context directory, and the file names, can be controlled from the configuration file.

Build Policy

If a repo contains the require kube-ci files, then a build will be triggered if:

  • A commit to a branches matching a configurable regexp is pushed (specifically when a CheckSuite even is sent). (Unless runForBranch is set to "false")
  • A new tag is created (only if the "runForTag" annotation is set to "true")
  • A non-Draft PR is raised between branches in the existing repository. (a configuration option allows building of Draft PR branches too). - If a PR is from a remote repository a manual run must be triggered using /kube-ci.
  • If /kube-ci run command is issued by a member of an org that matches a configurable regexp.

Workflow Annotations.

You can use annotations to control a few aspects of workflow execution.

Selective running: kube-ci.qutics.com/runForBranch: defaults to "true", runs for branches (specifically triggered from github CheckRun events when commits are pushed) kube-ci.qutics.com/runForTag: defaults to "false", runs when a tag is created.

Cache volume:

  • kube-ci.qutics.com/cacheScope: "project" or "branch", whether the volume is create per github project, or per branch. Branch cache volumes are deleted when the branch is deleted. All related volumes are deleted when a project is deleted or archived.
  • kube-ci.qutics.com/cacheSize: e.g. "20Gi", this can be used to set the size of the volume.
  • kube-ci.qutics.com/cacheStorageClassName: override the storage class name used for creating the volume
  • kube-ci.qutics.com/cacheName: This can be used to set a specific PVC claim name to be used as the cache, other cache settings will be ignored.

Manual step, and deployment controls:

  • kube-ci.qutics.com/manualTemplates: override the set of templates that will be presented as manually run check-run steps.
  • kube-ci.qutics.com/essentialTemplates: override the set of manual templates that will be fail the check until run (they are marked as conclusion action_required)
  • kube-ci.qutics.com/deployTemplates: override the set of templates that will be tracked via a GitHub deployment.
  • kube-ci.qutics.com/nonInteractiveBranches: Do not present manual check-run steps for check-run's created for commits with this as a the head branch. (these are in addition to check-runs created with the repositories default branch as the head branch.

Workflow Parameters.

Some extra parameteres will be added to your workflow before it is run.

  • repo: git@github.com:yourorg/repo.git
  • repo_git_url: git//github.com/yourorg/repo.git
  • repo_https_url: http://github.com/yourorg/repo.git
  • repoName: repo
  • orgName: yourorg
  • revision: 01245789abc.....
  • refType: "branch" or "tag"
  • refName: someref (regardless of branch or tag)
  • branch: newpr5 (the head branch) (only if this is a branch)
  • tag: v1.0.1 (tag if this is from a tag push or create)
  • cacheVolumeClaimName: the name of the pvc created (if requested)
  • repoDefaultBranch: the default branch configured on the repository

Workflows run for pull-requests will get this additional parameters:

  • pullRequestID: "123"
  • pullRequestBaseBranch: prbranch (The base branch being PR'd to

Workflow volumes.

You can create a volume to be used with a workflow by adding some annotations to your workflow.

annotations:
  "kube-ci.qutics.com/cacheScope": "project"
  "kube-ci.qutics.com/cacheSize": "20Gi"

TODO

  • better metrics
  • example deployment assets

Manual Check Run steps

On successful completion of the default template of your workflow, kube-ci can present an extra set of check-runs, set to "neutral". The user can then run, or skip those steps using buttons on the check-run. If the user clicks "Run", a new argo-workflow will be run for that step.

Manual check run steps default to "neutral" conclusion, which present as grey, and will not mark the check as failing. If you want to force a given template to be run, you can set the essentialTemplates configuration , or override the value using the workflow annotation.

Check runs are not presented in the following circumstances:

  • If the default entrypoint workflow fails
  • If the workflow that has run is not the default entrypoint workflow.
  • If the workflow head branch is the same as the repo default branch
  • If the workflow head branch matches the configured non-interactive-branches regular expression.

The last two steps are intended to prevent "failing" check-runs created for merge commits for a successfully merged PR (the logic here may need to change in future).

GitHub Deployments

Kube-CI can create Github Deployments to track which commits are deployed to which environments. Any step that includes a template that matches the configure DeploymentTemplates regular expression (or workflow annotation override), will have a Deployment created and tracked, based on the success/failure of the run of that deployment.

The template must include a parameter that passes in the specific environment to create the Deployment for. By default we look for environment in the parameters list, but this can be overriden with the environmentParameter configuration variable.

Deploying

  • Pick a URL for the webhook.The Kube-CI github WebHook needs to be reachable publicly by GitHub. e.g. at https://kube-ci.example.com/webhooks/github

  • Create a GitHub application

    • Set a webhook secret, keep a note of it.
    • Note the AppID.
    • Generate a private key. Download it somewhere.
    • Install the App on one of more accounts, you will need to take a note of the Installation ID (this shows up in the URL of the settings for the installation e.g. https://github.com/organizations/YOURORG/settings/installations/12345
    • It need Read permission to repos and repo contents, and read/write to Checks.
    • Enable the following events (TODO: not all of these are used, needs review):
      • Check suite
      • Check run
      • Push
      • Create
      • Delete
      • Commit Comment
      • Deployment
      • Deployment Status
      • Issue Comment
      • Pull Request
      • Pull Request Review
      • Release
      • Repository

  • Deploy kube-ci

    • You should already have argo deployed. Some things currently assume a namespace of "argo" exists for deploying the workflows to.
    • I've only tested this in cluster, but the flags are there for providing a kube-config if you want one.
    • You may need to add RBAC rules along the lines of those in rbac.yml
    • Setup a service/ingress for the webhook.
    • The follow options can be configured
      • -github.appid=1234
      • -config=/file.yml: path to yaml config file with some workflow defaults
      • -keyfile=/keyfile.pem: path to the secret key
      • `-secretfile=/webhook-secret": path to a file containing the webhook secret
      • "-argo.ui.base=https://kube-ci.example.com": external URL of your argo UI.
      • "-idsfile=/config/ids": not required, but limits the app to respond only to events for specific install IDs. One ID per line

  • At present we deploy kube-ci behind an oauth2 proxy that passes the require webhook URLs through, but requires auth for everything else using the github app info. All other traffic is proxied to the argo-ui. In future this will be baked in to kube-ci directly.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DefaultMain added in v0.9.2 

func DefaultMain()

func NewAPIHandler added in v0.10.0 

func NewAPIHandler(h APIHandler) http.Handler

func NewSlack added in v0.9.1 

func NewSlack(tokenFile, signingSecretFile string, ciFilePath string, templates TemplateSet) (*slackCmds, error)

func NewWorkflowSyner added in v0.9.1 

func NewWorkflowSyner(
	kubeclient kubernetes.Interface,
	clientset clientset.Interface,
	sinf informers.SharedInformerFactory,
	storage storageManager,
	ghClientSrc githubClientSource,
	appID int64,
	ghSecret []byte,
	baseURL string,
	config Config,
) *workflowSyncer

func StatusUpdate added in v0.9.1 

func StatusUpdate(
	ctx context.Context,
	info *githubInfo,
	status GithubStatus,
)

Types

type APIConfig added in v0.10.0 

type APIConfig struct {
	DefaultInstallID int `yaml:"defaultInstallID"`
}

type APIHandler added in v0.10.0 

type APIHandler struct {
	Storage pvcManager
	Clients githubClientSource
	Runner  workflowRunner
	Slash   slashRunner

	UIBase       string
	GitHubSecret []byte
	AppID        int64
	InstallID    int
}

func (*APIHandler) HandleRun added in v0.10.0 

func (h *APIHandler) HandleRun(w http.ResponseWriter, r *http.Request)

type CacheSpec 

type CacheSpec struct {
	Scope            string `yaml:"scope"`
	Size             string `yaml:"size"`
	StorageClassName string `yaml:"storageClassName"`
}

CacheSpec lets you choose the default settings for a per-job cache volume.

type Config 

type Config struct {
	CIContextPath  string            `yaml:"ciContextPath"`
	CIYAMLFile     string            `yaml:"ciYAMLFile"`
	CIStarlarkFile string            `yaml:"ciStarlarkFile"`
	Namespace      string            `yaml:"namespace"`
	ManagedBy      string            `yaml:"managedBy"`
	Tolerations    []v1.Toleration   `yaml:"tolerations"`
	NodeSelector   map[string]string `yaml:"nodeSelector"`
	TemplateSet    TemplateSet       `yaml:"templates"`
	CacheDefaults  CacheSpec         `yaml:"cacheDefaults"`
	BuildDraftPRs  bool              `yaml:"buildDraftPRs"`
	BuildBranches  string            `yaml:"buildBranches"`

	ManualTemplates        string `yaml:"manualTemplates"`
	EssentialTemplates     string `yaml:"essentialTemplates"`
	DeployTemplates        string `yaml:"deployTemplates"`
	ProductionEnvironments string `yaml:"productionEnvironments"`
	EnvironmentParameter   string `yaml:"environmentParameter"`
	NonInteractiveBranches string `yaml:"nonInteractiveBranches"`

	ExtraParameters map[string]string `yaml:"extraParameters"`

	API APIConfig `yaml:"api"`
	// contains filtered or unexported fields
}

Config defines our configuration file format

func ReadConfig added in v0.9.1 

func ReadConfig(configfile, defaultNamespace string) (*Config, error)

type DeploymentPayload added in v0.9.1 

type DeploymentPayload struct {
	KubeCI KubeCIPayload `json:"kube_ci"`
}

type GithubClientInterface added in v0.9.1 

type GithubClientInterface interface {
	GetInstallID() int
	GetHTTPClient() *http.Client
	GetRef(ctx context.Context, ref string) (*github.Reference, error)
	CreateCheckRun(ctx context.Context, opts github.CreateCheckRunOptions) (*github.CheckRun, error)
	GetRepo(ctx context.Context) (*github.Repository, error)
	UpdateCheckRun(ctx context.Context, id int64, upd github.UpdateCheckRunOptions) (*github.CheckRun, error)
	CreateDeployment(ctx context.Context, req *github.DeploymentRequest) (*github.Deployment, error)
	CreateDeploymentStatus(ctx context.Context, id int64, req *github.DeploymentStatusRequest) (*github.DeploymentStatus, error)
	IsMember(ctx context.Context, user string) (bool, error)
	CreateFile(ctx context.Context, filepath string, opts *github.RepositoryContentFileOptions) error
	GetContents(ctx context.Context, filepath string, opts *github.RepositoryContentGetOptions) ([]*github.RepositoryContent, error)
	GetBranch(ctx context.Context, branch string) (*github.Branch, error)
	GetPullRequest(ctx context.Context, prid int) (*github.PullRequest, error)
	CreateIssueComment(ctx context.Context, issueID int, opts *github.IssueComment) error
}

type GithubEvent added in v0.11.0 

type GithubEvent interface {
	GetInstallation() *github.Installation
	GetRepo() *github.Repository
	GetSender() *github.User
}

GithubEvent is a copy of the type in cistarlark and can probably be replaced by that one in future

type GithubKeyStore added in v0.9.1 

type GithubKeyStore struct {
	BaseTransport http.RoundTripper
	AppID         int64
	IDs           []int
	Key           []byte
	Orgs          *regexp.Regexp
}

type GithubStatus added in v0.9.1 

type GithubStatus struct {
	Status     string
	Conclusion string
	DetailsURL string
	Actions    []*github.CheckRunAction

	// output
	Title       string
	Summary     string
	Text        string
	Annotations []*github.CheckRunAnnotation
}

GithubStatus is a pseudo-ugly mapping of github checkrun and deployment status into a common struct to make workflow running agnostic to the reason the workflow was launched

type HookHandler added in v0.9.1 

type HookHandler struct {
	Storage pvcManager
	Clients githubClientSource
	Runner  workflowRunner
	Slash   slashRunner

	UIBase       string
	GitHubSecret []byte
	AppID        int64
}

func (*HookHandler) LoggingWebhook added in v0.9.1 

func (h *HookHandler) LoggingWebhook(w http.ResponseWriter, r *http.Request) (int, string)

type K8sStorageManager added in v0.9.1 

type K8sStorageManager struct {
	Namespace  string
	ManagedBy  string
	KubeClient kubernetes.Interface
}

type KubeCIPayload added in v0.9.1 

type KubeCIPayload struct {
	Run              bool `json:"run"`               // should we run a workflow
	CreateDeployment bool `json:"create_deployment"` // if true, don't create a deployment

	// RefType and RefName avoid going to the github API if we already know what type of
	// ref the user requested
	RefType string `json:"ref_type"`
	RefName string `json:"ref_name"`
	SHA     string `json:"sha"`
}

DeploymentPayload will do something useful eventually

type RunRequest added in v0.10.0 

type RunRequest struct {
	Org        string            `json:"org"`
	Repo       string            `json:"repo"`
	Ref        string            `json:"ref"`
	RefType    string            `json:"ref_type,omitempty"`
	Hash       string            `json:"hash"`
	CIContext  map[string]string `json:"ci_context"`
	Entrypoint string            `json:"entrypoint"`
	Run        bool              `json:"run"`
}

type RunResp added in v0.10.0 

type RunResp struct {
	Workflow *workflow.Workflow `json:"workflow,omitempty"`
	Debug    string             `json:"debug,omitempty"`
}

type SlashHandler added in v0.9.1 

type SlashHandler struct {
	Runner         workflowRunner
	CIContextPath  string
	CIYAMLFile     string
	CIStarlarkFile string
	Templates      TemplateSet
}

type StatusHandler added in v0.9.1 

type StatusHandler struct {
	Status *int32
}

func (*StatusHandler) ServeHTTP added in v0.9.1 

func (h *StatusHandler) ServeHTTP(w http.ResponseWriter, r *http.Request)

func (*StatusHandler) Shutdown added in v0.9.1 

func (h *StatusHandler) Shutdown()

type TemplateSet 

type TemplateSet map[string]TemplateSpec

TemplateSet describes a set of templates

func (TemplateSet) Help added in v0.5.4 

func (ts TemplateSet) Help() string

type TemplateSpec 

type TemplateSpec struct {
	Description string `yaml:"description"`
	CI          string `yaml:"ci"`
	Deploy      string `yaml:"deploy"`
}

TemplateSpec gives the description, and location, of a set of config files for use by the setup slash command

type WorkflowContext added in v0.10.0 

type WorkflowContext struct {
	Repo        *github.Repository
	Entrypoint  string
	Ref         string
	RefType     string
	SHA         string
	ContextData map[string]string
	PRs         []*github.PullRequest
	Event       GithubEvent
}

WorkflowContext is a copy of the type in cistarlark and can probably be replaced by that one in future

Source Files

View all Source files

Directories

Path Synopsis
cmd
kube-ci

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.