health

Documentation

Overview

Package health provides a generic health checking framework. The health package works expvar style. By importing the package the debug server is getting a "/debug/health" endpoint that returns the current status of the application. If there are no errors, "/debug/health" will return a HTTP 200 status, together with an empty JSON reply "{}". If there are any checks with errors, the JSON reply will include all the failed checks, and the response will be have an HTTP 503 status.

A Check can either be run synchronously, or asynchronously. We recommend that most checks are registered as an asynchronous check, so a call to the "/debug/health" endpoint always returns immediately. This pattern is particularly useful for checks that verify upstream connectivity or database status, since they might take a long time to return/timeout.

Installing

To install health, just import it in your application:

import "github.com/docker/distribution/health"

You can also (optionally) import "health/api" that will add two convenience endpoints: "/debug/health/down" and "/debug/health/up". These endpoints add "manual" checks that allow the service to quickly be brought in/out of rotation.

import _ "github.com/docker/distribution/registry/health/api"

# curl localhost:5001/debug/health
{}
# curl -X POST localhost:5001/debug/health/down
# curl localhost:5001/debug/health
{"manual_http_status":"Manual Check"}

After importing these packages to your main application, you can start registering checks.

Registering Checks

The recommended way of registering checks is using a periodic Check. PeriodicChecks run on a certain schedule and asynchronously update the status of the check. This allows CheckStatus to return without blocking on an expensive check.

A trivial example of a check that runs every 5 seconds and shuts down our server if the current minute is even, could be added as follows:

func currentMinuteEvenCheck() error {
  m := time.Now().Minute()
  if m%2 == 0 {
    return errors.New("Current minute is even!")
  }
  return nil
}

health.RegisterPeriodicFunc("minute_even", currentMinuteEvenCheck, time.Second*5)

Alternatively, you can also make use of "RegisterPeriodicThresholdFunc" to implement the exact same check, but add a threshold of failures after which the check will be unhealthy. This is particularly useful for flaky Checks, ensuring some stability of the service when handling them.

health.RegisterPeriodicThresholdFunc("minute_even", currentMinuteEvenCheck, time.Second*5, 4)

The lowest-level way to interact with the health package is calling "Register" directly. Register allows you to pass in an arbitrary string and something that implements "Checker" and runs your check. If your method returns an error with nil, it is considered a healthy check, otherwise it will make the health check endpoint "/debug/health" start returning a 503 and list the specific check that failed.

Assuming you wish to register a method called "currentMinuteEvenCheck() error" you could do that by doing:

health.Register("even_minute", health.CheckFunc(currentMinuteEvenCheck))

CheckFunc is a convenience type that implements Checker.

Another way of registering a check could be by using an anonymous function and the convenience method RegisterFunc. An example that makes the status endpoint always return an error:

health.RegisterFunc("my_check", func() error {
 return Errors.new("This is an error!")
}))

Examples

You could also use the health checker mechanism to ensure your application only comes up if certain conditions are met, or to allow the developer to take the service out of rotation immediately. An example that checks database connectivity and immediately takes the server out of rotation on err:

updater = health.NewStatusUpdater()
 health.RegisterFunc("database_check", func() error {
  return updater.Check()
}))

conn, err := Connect(...) // database call here
if err != nil {
  updater.Update(errors.New("Error connecting to the database: " + err.Error()))
}

You can also use the predefined Checkers that come included with the health package. First, import the checks:

import "github.com/docker/distribution/health/checks

After that you can make use of any of the provided checks. An example of using a `FileChecker` to take the application out of rotation if a certain file exists can be done as follows:

health.Register("fileChecker", health.PeriodicChecker(checks.FileChecker("/tmp/disable"), time.Second*5))

After registering the check, it is trivial to take an application out of rotation from the console:

# curl localhost:5001/debug/health
{}
# touch /tmp/disable
# curl localhost:5001/debug/health
{"fileChecker":"file exists"}

You could also test the connectivity to a downstream service by using a "HTTPChecker", but ensure that you only mark the test unhealthy if there are a minimum of two failures in a row:

health.Register("httpChecker", health.PeriodicThresholdChecker(checks.HTTPChecker("https://www.google.pt"), time.Second*5, 2))

Index

• func CheckStatus() map[string]string
• func Handler(handler http.Handler) http.Handler
• func Register(name string, check Checker)
• func RegisterFunc(name string, check func() error)
• func RegisterPeriodicFunc(name string, period time.Duration, check CheckFunc)
• func RegisterPeriodicThresholdFunc(name string, period time.Duration, threshold int, check CheckFunc)
• func StatusHandler(w http.ResponseWriter, r *http.Request)
• type CheckFunc
  • func (cf CheckFunc) Check() error
• type Checker
  • func PeriodicChecker(check Checker, period time.Duration) Checker
  • func PeriodicThresholdChecker(check Checker, period time.Duration, threshold int) Checker
• type Registry
  • func NewRegistry() *Registry
  • func (registry *Registry) CheckStatus() map[string]string
  • func (registry *Registry) Register(name string, check Checker)
  • func (registry *Registry) RegisterFunc(name string, check func() error)
  • func (registry *Registry) RegisterPeriodicFunc(name string, period time.Duration, check CheckFunc)
  • func (registry *Registry) RegisterPeriodicThresholdFunc(name string, period time.Duration, threshold int, check CheckFunc)
• type Updater
  • func NewStatusUpdater() Updater
  • func NewThresholdStatusUpdater(t int) Updater

Functions

func CheckStatus

func CheckStatus() map[string]string

CheckStatus returns a map with all the current health check errors from the default registry.

func Handler

func Handler(handler http.Handler) http.Handler

Handler returns a handler that will return 503 response code if the health checks have failed. If everything is okay with the health checks, the handler will pass through to the provided handler. Use this handler to disable a web application when the health checks fail.

func Register

func Register(name string, check Checker)

Register associates the checker with the provided name in the default registry.

func RegisterFunc

func RegisterFunc(name string, check func() error)

RegisterFunc allows the convenience of registering a checker in the default registry directly from an arbitrary func() error.

func RegisterPeriodicFunc

func RegisterPeriodicFunc(name string, period time.Duration, check CheckFunc)

RegisterPeriodicFunc allows the convenience of registering a PeriodicChecker in the default registry from an arbitrary func() error.

func RegisterPeriodicThresholdFunc

func RegisterPeriodicThresholdFunc(name string, period time.Duration, threshold int, check CheckFunc)

RegisterPeriodicThresholdFunc allows the convenience of registering a PeriodicChecker in the default registry from an arbitrary func() error.

func StatusHandler

func StatusHandler(w http.ResponseWriter, r *http.Request)

StatusHandler returns a JSON blob with all the currently registered Health Checks and their corresponding status. Returns 503 if any Error status exists, 200 otherwise

Types

type CheckFunc

type CheckFunc func() error

CheckFunc is a convenience type to create functions that implement the Checker interface

func (CheckFunc) Check

func (cf CheckFunc) Check() error

Check Implements the Checker interface to allow for any func() error method to be passed as a Checker

type Checker

type Checker interface {
	// Check returns nil if the service is okay.
	Check() error
}

Checker is the interface for a Health Checker

func PeriodicChecker

func PeriodicChecker(check Checker, period time.Duration) Checker

PeriodicChecker wraps an updater to provide a periodic checker

func PeriodicThresholdChecker

func PeriodicThresholdChecker(check Checker, period time.Duration, threshold int) Checker

PeriodicThresholdChecker wraps an updater to provide a periodic checker that uses a threshold before it changes status

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

A Registry is a collection of checks. Most applications will use the global registry defined in DefaultRegistry. However, unit tests may need to create separate registries to isolate themselves from other tests.

var DefaultRegistry *Registry

DefaultRegistry is the default registry where checks are registered. It is the registry used by the HTTP handler.

func NewRegistry

func NewRegistry() *Registry

NewRegistry creates a new registry. This isn't necessary for normal use of the package, but may be useful for unit tests so individual tests have their own set of checks.

func (*Registry) CheckStatus

func (registry *Registry) CheckStatus() map[string]string

CheckStatus returns a map with all the current health check errors

func (*Registry) Register

func (registry *Registry) Register(name string, check Checker)

Register associates the checker with the provided name.

func (*Registry) RegisterFunc

func (registry *Registry) RegisterFunc(name string, check func() error)

RegisterFunc allows the convenience of registering a checker directly from an arbitrary func() error.

func (*Registry) RegisterPeriodicFunc

func (registry *Registry) RegisterPeriodicFunc(name string, period time.Duration, check CheckFunc)

RegisterPeriodicFunc allows the convenience of registering a PeriodicChecker from an arbitrary func() error.

func (*Registry) RegisterPeriodicThresholdFunc

func (registry *Registry) RegisterPeriodicThresholdFunc(name string, period time.Duration, threshold int, check CheckFunc)

RegisterPeriodicThresholdFunc allows the convenience of registering a PeriodicChecker from an arbitrary func() error.

type Updater

type Updater interface {
	Checker

	// Update updates the current status of the health check.
	Update(status error)
}

Updater implements a health check that is explicitly set.

func NewStatusUpdater

func NewStatusUpdater() Updater

NewStatusUpdater returns a new updater

func NewThresholdStatusUpdater

func NewThresholdStatusUpdater(t int) Updater

NewThresholdStatusUpdater returns a new thresholdUpdater