kubernary

README

kubernary

Kubernary is a canary app for Kubernetes (or anywhere really).

Kubernary runs arbitrary checks on a regular interval, or on demand. These checks can emulate the common work patterns of real services inside your cluster and emit metrics. This can give you an early warning when overall cluster functionality is degraded.

Usage

Kubernary runs as a daemon. It runs all checks at a configurable interval. Checks generally emit logs and metrics at run time.

$ docker run negz/kubernary /kubernary/kubernary --help
usage: kubernary [<flags>] <statsd>

Checks whether your Kubernetes cluster works.

Flags:
      --help             Show context-sensitive help (also try --help-long and
                         --help-man).
      --no-stats         Don't send statsd stats.
      --listen=":10002"  Address at which to expose HTTP health checks.
  -d, --debug            Run with debug logging.
      --close-after=1m   Wait this long at shutdown before closing HTTP
                         connections.
      --kill-after=2m    Wait this long at shutdown before exiting.

Args:
  <statsd>  Address to which to send statsd metrics.

Kubernary exposes the following HTTP endpoints:

• http://kubernary/quitquitquit - Causes Kubernary to shutdown and exit immediately.
• http://kubernary/health - Runs all checks on-demand.

The /health endpoint returns:

• 200 OK - If all checks pass.
• 503 SERVICE UNAVAILABLE - If one or more check fails.
• 500 INTERNAL SERVER ERROR - If an error unrelated to a check occurs.

Along with the following JSON body:

{
  "s3": {
    "ok": true,
    "error": ""
  },
  "failingcheck": {
    "ok": false,
    "error": "Kaboom!"
  }
}

Checks

Currently the only check is Amazon S3. This check ensures a Kubernary can read a file from an S3 bucket, primarily as a way of validating that kube2iam is functioning correctly in a Kubernetes cluster.

The check uses the following environment variables for configuration:

• KUBERNARY_S3_BUCKET - The bucket to read from.
• KUBERNARY_S3_KEY - The key to read within the bucket. Reading a very small or zero length file is recommended.

The following statsd metrics are emitted by the check:

• kubernary.s3.download.succeeded - A count of successful S3 downloads.
• kubernary.s3.download.failed - A count of failed S3 downloads.

Building

To build a Docker image run the following with a working Go environment:

$ scripts/build.sh

This (possibly somewhat brittle) script will build a Kubernary binary, and copy it into a Docker image. You'll need to push it to ECR manually:

VERSION=$(git rev-parse -short HEAD)

docker push negz/kubernary:latest
docker push negz/kubernary:${VERSION}

Documentation

Index

• Constants
• func CheckConfigFromEnv(name string, config map[string]string) map[string]string
• func CheckHandler(cfg *CheckConfig) http.HandlerFunc
• func ChecksHandler(cfgs []*CheckConfig) http.HandlerFunc
• func RunCheckForever(cfg *CheckConfig) context.CancelFunc
• func RunChecksForever(cfgs []*CheckConfig) context.CancelFunc
• func ShutdownHandler(cancel context.CancelFunc) http.HandlerFunc
• type CheckConfig
• type Checker

Constants

const CheckConfigEnvPrefix string = "KUBERNARY_"

CheckConfigEnvPrefix is the prefix required by any check configuration environment variables.

Functions

func CheckConfigFromEnv

func CheckConfigFromEnv(name string, config map[string]string) map[string]string

CheckConfigFromEnv provides a standard pattern for reading basic check specific config from environment variables. Pass in a map of keys with default values. If KUBERNARY_CHECKNAME_KEY is set for key its default value will be overwritten.

func CheckHandler

func CheckHandler(cfg *CheckConfig) http.HandlerFunc

CheckHandler returns an HTTP handler that runs the provided check and returns the results.

func ChecksHandler

func ChecksHandler(cfgs []*CheckConfig) http.HandlerFunc

ChecksHandler returns an HTTP handler that runs the provided checks concurrently and returns the results.

func RunCheckForever

func RunCheckForever(cfg *CheckConfig) context.CancelFunc

RunCheckForever causes a check to be run every configured interval, forever.

func RunChecksForever

func RunChecksForever(cfgs []*CheckConfig) context.CancelFunc

RunChecksForever causes a slice of checks to be run every configured interval, forever.

func ShutdownHandler

func ShutdownHandler(cancel context.CancelFunc) http.HandlerFunc

ShutdownHandler shuts down kubernary when called.

Types

type CheckConfig

type CheckConfig struct {
	Checker  Checker
	Interval time.Duration
	Timeout  time.Duration
}

A CheckConfig specified how a check should be run.

type Checker

type Checker interface {
	Check() error
	Name() string
}

A Checker is a health check run by kubernary to assess cluster functionality.