README
¶
Prow
Prow is a Kubernetes based CI/CD system. Jobs can be triggered by various types of events and report their status to many different services. In addition to job execution, Prow provides GitHub automation in the form of policy enforcement, chat-ops via /foo style commands, and automatic PR merging.
See the GoDoc for library docs.
Please note that these libraries are intended for use by prow only, and we do
not make any attempt to preserve backwards compatibility.
For a brief overview of how Prow runs jobs take a look at "Life of a Prow Job".
Functions and Features
-
Job execution for testing, batch processing, artifact publishing.
- GitHub events are used to trigger post-PR-merge (postsubmit) jobs and on-PR-update (presubmit) jobs.
- Support for multiple execution platforms and source code review sites.
-
Pluggable GitHub bot automation that implements
/foostyle commands and enforces configured policies/processes. -
GitHub merge automation with batch testing logic.
-
Front end for viewing jobs, merge queue status, dynamically generated help information, and more.
-
Automatic deployment of source control based config.
-
Automatic GitHub org/repo administration configured in source control.
-
Designed for multi-org scale with dozens of repositories. (The Kubernetes Prow instance uses only 1 GitHub bot token!)
-
High availability as benefit of running on Kubernetes. (replication, load balancing, rolling updates...)
-
JSON structured logs.
-
Prometheus metrics.
Documentation
Getting started
- With your own Prow deployment:
getting_started_deploy.md - With developing for Prow:
getting_started_develop.md - As a job author:
jobs.md
More details
- Components
- Plugins
- ProwJobs
- Building, Testing, and Updating
- General Configuration
- Pod Utilities
- Scaling Prow
- Tide
- Metrics
- "Life of a Prow Job"
- Getting more out of Prow
Useful Talks
KubeCon 2018 EU
Automation and the Kubernetes Contributor Experience SIG Testing Deep Dive
Prow in the wild
Prow is used by the following organizations and projects:
- Kubernetes
- This includes kubernetes, kubernetes-client, kubernetes-csi, kubernetes-incubator, and kubernetes-sigs.
- OpenShift
- This includes openshift, openshift-s2i, operator-framework, and some repos in kubernetes-incubator, containers and heketi.
- Istio
- Knative
- Jetstack
- Kyma
- Prometheus
- Caicloud
- Kubeflow
- Azure acs-engine
- tensorflow/minigo
- helm/charts
Jenkins X uses Prow as part of Serverless Jenkins.
Contact us
If you need to contact the maintainers of Prow you have a few options:
- Open an issue in the kubernetes/test-infra repo.
- Reach out to the
#prowchannel of the Kubernetes Slack. - Contact one of the code owners in prow/OWNERS or in a more specifically scoped OWNERS file.
Bots home
@k8s-ci-robot lives here and is the face of the Kubernetes Prow instance. Here is a command list for interacting with @k8s-ci-robot and other Prow bots.
Directories
¶
| Path | Synopsis |
|---|---|
|
apis
|
|
|
prowjobs/v1
Package v1 is the v1 version of the API.
|
Package v1 is the v1 version of the API. |
|
client
|
|
|
clientset/versioned
This package has the automatically generated clientset.
|
This package has the automatically generated clientset. |
|
clientset/versioned/fake
This package has the automatically generated fake clientset.
|
This package has the automatically generated fake clientset. |
|
clientset/versioned/scheme
This package contains the scheme of the automatically generated clientset.
|
This package contains the scheme of the automatically generated clientset. |
|
clientset/versioned/typed/prowjobs/v1
This package has the automatically generated typed clients.
|
This package has the automatically generated typed clients. |
|
clientset/versioned/typed/prowjobs/v1/fake
Package fake has the automatically generated clients.
|
Package fake has the automatically generated clients. |
|
Package clonerefs is a library for cloning references
|
Package clonerefs is a library for cloning references |
|
cmd
|
|
|
checkconfig
checkconfig loads configuration for Prow to validate it
|
checkconfig loads configuration for Prow to validate it |
|
gcsupload
gcsupload uploads the files and folders specified to GCS using the Prow-defined job configuration
|
gcsupload uploads the files and folders specified to GCS using the Prow-defined job configuration |
|
initupload
initupload parses the logs from the clonerefs container and determines if that container was successful or not.
|
initupload parses the logs from the clonerefs container and determines if that container was successful or not. |
|
tot
Tot vends (rations) incrementing numbers for use in builds.
|
Tot vends (rations) incrementing numbers for use in builds. |
|
tot/fallbackcheck
fallbackcheck reports whether jobs in the provided prow deployment have fallback build numbers in GCS.
|
fallbackcheck reports whether jobs in the provided prow deployment have fallback build numbers in GCS. |
|
Package commentpruner facilitates efficiently deleting bot comments as a reaction to webhook events.
|
Package commentpruner facilitates efficiently deleting bot comments as a reaction to webhook events. |
|
Package config knows how to read and parse config.yaml.
|
Package config knows how to read and parse config.yaml. |
|
secret
Package secret implements an agent to read and reload the secrets.
|
Package secret implements an agent to read and reload the secrets. |
|
Package crier reports finished prowjob status to git providers.
|
Package crier reports finished prowjob status to git providers. |
|
Package cron provides a wrapper of robfig/cron, which manages schedule cron jobs for horologium
|
Package cron provides a wrapper of robfig/cron, which manages schedule cron jobs for horologium |
|
deck
|
|
|
jobs
Package jobs implements methods on job information used by Prow component deck
|
Package jobs implements methods on job information used by Prow component deck |
|
Package entrypoint is a library that knows how to wrap a process and write it's output and exit code to disk
|
Package entrypoint is a library that knows how to wrap a process and write it's output and exit code to disk |
|
Package errorutil provides utilities for errors
|
Package errorutil provides utilities for errors |
|
external-plugins
|
|
|
refresh
Refresh retries Github status updates for stale PR statuses.
|
Refresh retries Github status updates for stale PR statuses. |
|
Package flagutil contains utilities and interfaces shared between several Prow commands.
|
Package flagutil contains utilities and interfaces shared between several Prow commands. |
|
Package gcsupload uploads artifacts to a GCS path resolved from job configuration
|
Package gcsupload uploads artifacts to a GCS path resolved from job configuration |
|
Package genfiles understands the .generated_files config file.
|
Package genfiles understands the .generated_files config file. |
|
gerrit
|
|
|
adapter
Package adapter implements a controller that interacts with gerrit instances
|
Package adapter implements a controller that interacts with gerrit instances |
|
client
Package client implements a client that can handle multiple gerrit instances derived from https://github.com/andygrunwald/go-gerrit
|
Package client implements a client that can handle multiple gerrit instances derived from https://github.com/andygrunwald/go-gerrit |
|
reporter
Package reporter implements a reporter interface for gerrit
|
Package reporter implements a reporter interface for gerrit |
|
Package git provides a client to plugins that can do git operations.
|
Package git provides a client to plugins that can do git operations. |
|
localgit
Package localgit creates a local git repo that can be used for testing code that uses a git.Client.
|
Package localgit creates a local git repo that can be used for testing code that uses a git.Client. |
|
reporter
Package reporter implements a reporter interface for github TODO(krzyzacy): move logic from report.go here
|
Package reporter implements a reporter interface for github TODO(krzyzacy): move logic from report.go here |
|
Package initupload determines the output status of clone operations and posts that status along with artifacts and logs to cloud storage.
|
Package initupload determines the output status of clone operations and posts that status along with artifacts and logs to cloud storage. |
|
Package jenkins includes a client and the operational logic for managing Jenkins masters in prow.
|
Package jenkins includes a client and the operational logic for managing Jenkins masters in prow. |
|
Package logrusutil implements some helpers for using logrus
|
Package logrusutil implements some helpers for using logrus |
|
Package metrics contains utilities for working with metrics in prow.
|
Package metrics contains utilities for working with metrics in prow. |
|
Package pjutil contains helpers for working with ProwJobs.
|
Package pjutil contains helpers for working with ProwJobs. |
|
Package pluginhelp defines structures that represent plugin help information.
|
Package pluginhelp defines structures that represent plugin help information. |
|
externalplugins
Package externalplugins provides the plugin help components to be compiled into external plugin binaries.
|
Package externalplugins provides the plugin help components to be compiled into external plugin binaries. |
|
hook
Package hook provides the plugin help components to be compiled into the hook binary.
|
Package hook provides the plugin help components to be compiled into the hook binary. |
|
blockade
Package blockade defines a plugin that adds the 'do-not-merge/blocked-paths' label to PRs that modify protected file paths.
|
Package blockade defines a plugin that adds the 'do-not-merge/blocked-paths' label to PRs that modify protected file paths. |
|
buildifier
Package buildifier defines a Prow plugin that runs buildifier over modified BUILD, WORKSPACE, and skylark (.bzl) files in pull requests.
|
Package buildifier defines a Prow plugin that runs buildifier over modified BUILD, WORKSPACE, and skylark (.bzl) files in pull requests. |
|
cat
Package cat adds cat images to an issue or PR in response to a /meow comment
|
Package cat adds cat images to an issue or PR in response to a /meow comment |
|
cherrypickunapproved
Package cherrypickunapproved adds the `do-not-merge/cherry-pick-not-approved` label to PRs against a release branch which do not have the `cherry-pick-approved` label.
|
Package cherrypickunapproved adds the `do-not-merge/cherry-pick-not-approved` label to PRs against a release branch which do not have the `cherry-pick-approved` label. |
|
dco
Package dco implements a DCO (https://developercertificate.org/) checker plugin
|
Package dco implements a DCO (https://developercertificate.org/) checker plugin |
|
docs-no-retest
Package docsnoretest contains a Prow plugin which manages a label indicating whether a given pull requests only changes documentation.
|
Package docsnoretest contains a Prow plugin which manages a label indicating whether a given pull requests only changes documentation. |
|
dog
Package dog adds dog images to the issue or PR in response to a /woof comment
|
Package dog adds dog images to the issue or PR in response to a /woof comment |
|
hold
Package hold contains a plugin which will allow users to label their own pull requests as not ready or ready for merge.
|
Package hold contains a plugin which will allow users to label their own pull requests as not ready or ready for merge. |
|
lgtm
Package lgtm implements the lgtm plugin
|
Package lgtm implements the lgtm plugin |
|
milestone
Package milestone implements the `/milestone` command which allows members of the milestone maintainers team to specify a milestone to be applied to an Issue or PR.
|
Package milestone implements the `/milestone` command which allows members of the milestone maintainers team to specify a milestone to be applied to an Issue or PR. |
|
milestonestatus
Package milestonestatus implements the `/status` command which allows members of the milestone maintainers team to specify a `status/*` label to be applied to an Issue or PR.
|
Package milestonestatus implements the `/status` command which allows members of the milestone maintainers team to specify a `status/*` label to be applied to an Issue or PR. |
|
override
Package override supports the /override context command.
|
Package override supports the /override context command. |
|
pony
Package pony adds pony images to the issue or PR in response to a /pony comment
|
Package pony adds pony images to the issue or PR in response to a /pony comment |
|
require-matching-label
Package requirematchinglabel implements the `require-matching-label` plugin.
|
Package requirematchinglabel implements the `require-matching-label` plugin. |
|
sigmention
Package sigmention recognize SIG '@' mentions and adds 'sig/*' and 'kind/*' labels as appropriate.
|
Package sigmention recognize SIG '@' mentions and adds 'sig/*' and 'kind/*' labels as appropriate. |
|
size
Package size contains a Prow plugin which counts the number of lines changed in a pull request, buckets this number into a few size classes (S, L, XL, etc), and finally labels the pull request with this size.
|
Package size contains a Prow plugin which counts the number of lines changed in a pull request, buckets this number into a few size classes (S, L, XL, etc), and finally labels the pull request with this size. |
|
skip
Package skip implements the `/skip` command which allows users to clean up commit statuses of non-blocking presubmits on PRs.
|
Package skip implements the `/skip` command which allows users to clean up commit statuses of non-blocking presubmits on PRs. |
|
stage
Package stage defines a Prow plugin that defines the stage of the issue in the features process.
|
Package stage defines a Prow plugin that defines the stage of the issue in the features process. |
|
welcome
Package welcome implements a prow plugin to welcome new contributors
|
Package welcome implements a prow plugin to welcome new contributors |
|
wip
Package wip will label a PR a work-in-progress if the author provides a prefix to their pull request title to the same effect.
|
Package wip will label a PR a work-in-progress if the author provides a prefix to their pull request title to the same effect. |
|
pod-utils
|
|
|
decorate
Package decorate is a library for adding to a user-provided PodSpec in order to create a full Pod that will fulfill a test job
|
Package decorate is a library for adding to a user-provided PodSpec in order to create a full Pod that will fulfill a test job |
|
downwardapi
Package downwardapi declares the types used to expose job configuration to the jobs themselves
|
Package downwardapi declares the types used to expose job configuration to the jobs themselves |
|
gcs
Package gcs handles uploading files and raw data to GCS and determines where in the GCS bucket data should go given a specific job specification
|
Package gcs handles uploading files and raw data to GCS and determines where in the GCS bucket data should go given a specific job specification |
|
options
Package options abstracts the options loading flow for pod utilities
|
Package options abstracts the options loading flow for pod utilities |
|
wrapper
Package wrapper contains utilities for the processes that wrap the test execution in a ProwJob test container
|
Package wrapper contains utilities for the processes that wrap the test execution in a ProwJob test container |
|
pubsub
|
|
|
reporter
Package reporter contains helpers for publishing statues to Pub statuses in Github.
|
Package reporter contains helpers for publishing statues to Pub statuses in Github. |
|
Package report contains helpers for writing comments and updating statuses in Github.
|
Package report contains helpers for writing comments and updating statuses in Github. |
|
Package sidecar is a library that knows how to report on the output of a process that writes its output and exit code to disk
|
Package sidecar is a library that knows how to report on the output of a process that writes its output and exit code to disk |
|
Package spyglass creates views for Prow job artifacts.
|
Package spyglass creates views for Prow job artifacts. |
|
lenses
Package lenses provides interfaces and methods necessary for implementing custom artifact viewers
|
Package lenses provides interfaces and methods necessary for implementing custom artifact viewers |
|
lenses/buildlog
Package buildlog provides a build log viewer for Spyglass
|
Package buildlog provides a build log viewer for Spyglass |
|
lenses/junit
Package junit provides a junit viewer for Spyglass
|
Package junit provides a junit viewer for Spyglass |
|
lenses/metadata
Package metadata provides a metadata viewer for Spyglass
|
Package metadata provides a metadata viewer for Spyglass |
|
Package statusreconciler ensures that changes to required presubmits do not cause PRs in flight to get stuck in the merge queue
|
Package statusreconciler ensures that changes to required presubmits do not cause PRs in flight to get stuck in the merge queue |
|
Package tide contains a controller for managing a tide pool of PRs.
|
Package tide contains a controller for managing a tide pool of PRs. |
|
history
Package history provides an append only, size limited log of recent actions that Tide has taken for each subpool.
|
Package history provides an append only, size limited log of recent actions that Tide has taken for each subpool. |
Click to show internal directories.
Click to hide internal directories.