skipper

README

Skipper

Skipper is an HTTP router and reverse proxy for service composition. It's designed to handle >100k HTTP route definitions with detailed lookup conditions, and flexible augmentation of the request flow with filters. It can be used out of the box or extended with custom lookup, filter logic and configuration sources.

NOTE for Skoap users

The Skoap filters can be found currently in the branch called 'skoap-migration'. The original incubator repository at zalando-incubator/skoap has been removed.

Main features:

• identifies routes based on the requests' properties, such as path, method, host and headers
• allows modification of the requests and responses with filters that are independently configured for each route
• simultaneously streams incoming requests and backend responses
• optionally acts as a final endpoint (shunt), e.g. as a static file server or a mock backend for diagnostics
• updates routing rules without downtime, while supporting multiple types of data sources — including etcd, Innkeeper, static files and custom configuration sources
• can serve as a Kubernetes Ingress implementation in combination with a controller that will route public traffic to your skipper fleet; see AWS example
• shipped with eskip: a descriptive configuration language designed for routing rules

Skipper provides a default executable command with a few built-in filters. However, its primary use case is to be extended with custom filters, predicates or data sources. Go here for additional documentation.

A few examples for extending Skipper:

• Authentication proxy https://github.com/zalando-incubator/skoap (repository removed see 'skoap-migration' branch)
• Image server https://github.com/zalando-stups/skrop

Getting Started

Prerequisites/Requirements

In order to build and run Skipper, only the latest version of Go needs to be installed. Skipper can use Innkeeper or Etcd as data sources for routes, or for the simplest cases, a local configuration file. See more details in the documentation: https://godoc.org/github.com/zalando/skipper.

Installation

Skipper is 'go get' compatible. If needed, create a Go workspace first:

mkdir ws
cd ws
export GOPATH=$(pwd)
export PATH=$PATH:$GOPATH/bin

Get the Skipper packages:

go get github.com/zalando/skipper/...

Running

Create a file with a route:

echo 'hello: Path("/hello") -> "https://www.example.org"' > example.eskip

Optionally, verify the file's syntax:

eskip check example.eskip

Start Skipper and make an HTTP request:

skipper -routes-file example.eskip &
curl localhost:9090/hello

Docker

To run the latest Docker container:

docker run registry.opensource.zalan.do/pathfinder/skipper:latest

Working with the code

Getting the code with the test dependencies (-t switch):

go get -t github.com/zalando/skipper/...

Build and test all packages:

cd src/github.com/zalando/skipper
make deps
make install
make check

Kubernetes Ingress

Skipper can be used to run as an Ingress implementation in combination with a controller, e.g. https://github.com/zalando-incubator/kube-ingress-aws-controller. A production example, https://github.com/zalando-incubator/kubernetes-on-aws/blob/dev/cluster/manifests/skipper/daemonset.yaml, can be found in our Kubernetes configuration https://github.com/zalando-incubator/kubernetes-on-aws.

Documentation

Skipper's Godoc page, https://godoc.org/github.com/zalando/skipper, includes detailed information on these topics:

• The Routing Mechanism
• Matching Requests
• Filters - Augmenting Requests
• Service Backends
• Route Definitions
• Data Sources: Kubernetes
• Circuit Breakers
• Extending It with Customized Predicates, Filters, and Builds
• Proxy Packages
• Logging and Metrics
• Performance Considerations
• Rate Limiters

Packaging support

See https://github.com/zalando/skipper/blob/master/packaging/readme.md

In case you want to implement and link your own modules into your skipper for more advanced features like opentracing API support there is https://github.com/skipper-plugins organization to enable you to do so. In order to explain you the build process with custom Go modules there is https://github.com/skipper-plugins/skipper-tracing-build, that is used to build skipper's opentracing package.

Community

User or developer questions can be asked in our public Google Group

Proposals

We do our proposals open in Skipper's Google drive. If you want to make a proposal feel free to create an issue and if it is a bigger change we will invite you to a document, such that we can work together.

Documentation

Overview

Package skipper provides an HTTP routing library with flexible configuration as well as a runtime update of the routing rules.

Skipper works as an HTTP reverse proxy that is responsible for mapping incoming requests to multiple HTTP backend services, based on routes that are selected by the request attributes. At the same time, both the requests and the responses can be augmented by a filter chain that is specifically defined for each route. Optionally, it can provide circuit breaker mechanism individually for each backend host.

Skipper can load and update the route definitions from multiple data sources without being restarted.

It provides a default executable command with a few built-in filters, however, its primary use case is to be extended with custom filters, predicates or data sources. For futher information read 'Extending Skipper'.

Skipper took the core design and inspiration from Vulcand: https://github.com/mailgun/vulcand.

Quickstart

Skipper is 'go get' compatible. If needed, create a 'go workspace' first:

mkdir ws
cd ws
export GOPATH=$(pwd)
export PATH=$PATH:$GOPATH/bin

Get the Skipper packages:

go get github.com/zalando/skipper/...

Create a file with a route:

echo 'hello: Path("/hello") -> "https://www.example.org"' > example.eskip

Optionally, verify the syntax of the file:

eskip check example.eskip

Start Skipper and make an HTTP request:

skipper -routes-file example.eskip &
curl localhost:9090/hello

Routing Mechanism

The core of Skipper's request processing is implemented by a reverse proxy in the 'proxy' package. The proxy receives the incoming request, forwards it to the routing engine in order to receive the most specific matching route. When a route matches, the request is forwarded to all filters defined by it. The filters can modify the request or execute any kind of program logic. Once the request has been processed by all the filters, it is forwarded to the backend endpoint of the route. The response from the backend goes once again through all the filters in reverse order. Finally, it is mapped as the response of the original incoming request.

Besides the default proxying mechanism, it is possible to define routes without a real network backend endpoint. One of these cases is called a 'shunt' backend, in which case one of the filters needs to handle the request providing its own response (e.g. the 'static' filter). Actually, filters themselves can instruct the request flow to shunt by calling the Serve(*http.Response) method of the filter context.

Another case of a route without a network backend is the 'loopback'. A loopback route can be used to match a request, modified by filters, against the lookup tree with different conditions and then execute a different route. One example scenario can be to use a single route as an entry point to execute some calculation to get an A/B testing decision and then matching the updated request metadata for the actual destination route. This way the calculation can be executed for only those requests that don't contain information about a previously calculated decision.

For further details, see the 'proxy' and 'filters' package documentation.

Matching Requests

Finding a request's route happens by matching the request attributes to the conditions in the route's definitions. Such definitions may have the following conditions:

- method

- path (optionally with wildcards)

- path regular expressions

- host regular expressions

- headers

- header regular expressions

It is also possible to create custom predicates with any other matching criteria.

The relation between the conditions in a route definition is 'and', meaning, that a request must fulfill each condition to match a route.

For further details, see the 'routing' package documentation.

Filters - Augmenting Requests

Filters are applied in order of definition to the request and in reverse order to the response. They are used to modify request and response attributes, such as headers, or execute background tasks, like logging. Some filters may handle the requests without proxying them to service backends. Filters, depending on their implementation, may accept/require parameters, that are set specifically to the route.

For further details, see the 'filters' package documentation.

Service Backends

Each route has one of the following backends: HTTP endpoint, shunt or loopback.

Backend endpoints can be any HTTP service. They are specified by their network address, including the protocol scheme, the domain name or the IP address, and optionally the port number: e.g. "https://www.example.org:4242". (The path and query are sent from the original request, or set by filters.)

A shunt route means that Skipper handles the request alone and doesn't make requests to a backend service. In this case, it is the responsibility of one of the filters to generate the response.

A loopback route executes the routing mechanism on current state of the request from the start, including the route lookup. This way it serves as a form of an internal redirect.

Route Definitions

Route definitions consist of the following:

- request matching conditions (predicates)

- filter chain (optional)

- backend (either an HTTP endpoint or a shunt)

The eskip package implements the in-memory and text representations of route definitions, including a parser.

(Note to contributors: in order to stay compatible with 'go get', the generated part of the parser is stored in the repository. When changing the grammar, 'go generate' needs to be executed explicitly to update the parser.)

For further details, see the 'eskip' package documentation

Data Sources

Skipper's route definitions of Skipper are loaded from one or more data sources. It can receive incremental updates from those data sources at runtime. It provides three different data clients:

- Kubernetes: Skipper can be used as part of a Kubernetes Ingress Controller implementation together with https://github.com/zalando-incubator/kube-ingress-aws-controller . In this scenario, Skipper uses the Kubernetes API's Ingress extensions as a source for routing. For a complete deployment example, see more details in: https://github.com/zalando-incubator/kubernetes-on-aws/ .

- Innkeeper: the Innkeeper service implements a storage for large sets of Skipper routes, with an HTTP+JSON API, OAuth2 authentication and role management. See the 'innkeeper' package and https://github.com/zalando/innkeeper.

- etcd: Skipper can load routes and receive updates from etcd clusters (https://github.com/coreos/etcd). See the 'etcd' package.

- static file: package eskipfile implements a simple data client, which can load route definitions from a static file in eskip format. Currently, it loads the routes on startup. It doesn't support runtime updates.

Skipper can use additional data sources, provided by extensions. Sources must implement the DataClient interface in the routing package.

Circuit Breaker

Skipper provides circuit breakers, configured either globally, based on backend hosts or based on individual routes. It supports two types of circuit breaker behavior: open on N consecutive failures, or open on N failures out of M requests. For details, see: https://godoc.org/github.com/zalando/skipper/circuit.

Running Skipper

Skipper can be started with the default executable command 'skipper', or as a library built into an application. The easiest way to start Skipper as a library is to execute the 'Run' function of the current, root package.

Each option accepted by the 'Run' function is wired in the default executable as well, as a command line flag. E.g. EtcdUrls becomes -etcd-urls as a comma separated list. For command line help, enter:

skipper -help

An additional utility, eskip, can be used to verify, print, update and delete routes from/to files or etcd (Innkeeper on the roadmap). See the cmd/eskip command package, and/or enter in the command line:

eskip -help

Extending Skipper

Skipper doesn't use dynamically loaded plugins, however, it can be used as a library, and it can be extended with custom predicates, filters and/or custom data sources.

Custom Predicates

To create a custom predicate, one needs to implement the PredicateSpec interface in the routing package. Instances of the PredicateSpec are used internally by the routing package to create the actual Predicate objects as referenced in eskip routes, with concrete arguments.

Example, randompredicate.go:

package main

import (
    "github.com/zalando/skipper/routing"
    "math/rand"
    "net/http"
)

type randomSpec struct {}

type randomPredicate struct {
    chance float64
}

func (s *randomSpec) Name() string { return "Random" }

func (s *randomSpec) Create(args []interface{}) routing.Predicate {
    p := &randomPredicate{.5}
    if len(args) > 0 {
        if c, ok := args[0].(float64); ok {
            p.chance = c
        }
    }

    return p
}

func (p *randomPredicate) Match(_ *http.Request) bool {
    return rand.Float64() < p.chance
}

In the above example, a custom predicate is created, that can be referenced in eskip definitions with the name 'Random':

Random(.33) -> "https://test.example.org";
* -> "https://www.example.org"

Custom Filters

To create a custom filter we need to implement the Spec interface of the filters package. 'Spec' is the specification of a filter, and it is used to create concrete filter instances, while the raw route definitions are processed.

Example, hellofilter.go:

package main

import (
    "fmt"
    "github.com/zalando/skipper/filters"
)

type helloSpec struct {}

type helloFilter struct {
    who string
}

func (s *helloSpec) Name() string { return "hello" }

func (s *helloSpec) CreateFilter(config []interface{}) (filters.Filter, error) {
    if len(config) == 0 {
        return nil, filters.ErrInvalidFilterParameters
    }

    if who, ok := config[0].(string); ok {
        return &helloFilter{who}, nil
    } else {
        return nil, filters.ErrInvalidFilterParameters
    }
}

func (f *helloFilter) Request(ctx filters.FilterContext) {}

func (f *helloFilter) Response(ctx filters.FilterContext) {
    ctx.Response().Header.Set("X-Hello", fmt.Sprintf("Hello, %s!", f.who))
}

The above example creates a filter specification, and in the routes where they are included, the filter instances will set the 'X-Hello' header for each and every response. The name of the filter is 'hello', and in a route definition it is referenced as:

• -> hello("world") -> "https://www.example.org"

Custom Build

The easiest way to create a custom Skipper variant is to implement the required filters (as in the example above) by importing the Skipper package, and starting it with the 'Run' command.

Example, hello.go:

package main

import (
    "github.com/zalando/skipper"
    "github.com/zalando/skipper/filters"
    "log"
)

func main() {
    log.Fatal(skipper.Run(skipper.Options{
        Address: ":9090",
        RoutesFile: "routes.eskip",
        CustomPredicates: []routing.PredicateSpec{&randomSpec{}},
        CustomFilters: []filters.Spec{&helloSpec{}}}))
}

A file containing the routes, routes.eskip:

Random(.05) -> hello("fish?") -> "https://fish.example.org";
* -> hello("world") -> "https://www.example.org"

Start the custom router:

go run hello.go

Proxy Package Used Individually

The 'Run' function in the root Skipper package starts its own listener but it doesn't provide the best composability. The proxy package, however, provides a standard http.Handler, so it is possible to use it in a more complex solution as a building block for routing.

Logging and Metrics

Skipper provides detailed logging of failures, and access logs in Apache log format. Skipper also collects detailed performance metrics, and exposes them on a separate listener endpoint for pulling snapshots.

For details, see the 'logging' and 'metrics' packages documentation.

Performance Considerations

The router's performance depends on the environment and on the used filters. Under ideal circumstances, and without filters, the biggest time factor is the route lookup. Skipper is able to scale to thousands of routes with logarithmic performance degradation. However, this comes at the cost of increased memory consumption, due to storing the whole lookup tree in a single structure.

Benchmarks for the tree lookup can be run by:

go test github.com/zalando/skipper/routing -bench=Tree

In case more aggressive scale is needed, it is possible to setup Skipper in a cascade model, with multiple Skipper instances for specific route segments.

Index

• func Run(o Options) error
• type Options

Functions

func Run

func Run(o Options) error

Run skipper.

Types

type Options

type Options struct {

	// Network address that skipper should listen on.
	Address string

	// List of custom filter specifications.
	CustomFilters []filters.Spec

	// Urls of nodes in an etcd cluster, storing route definitions.
	EtcdUrls []string

	// Path prefix for skipper related data in the etcd storage.
	EtcdPrefix string

	// Timeout used for a single request when querying for updates
	// in etcd. This is independent of, and an addition to,
	// SourcePollTimeout. When not set, the internally defined 1s
	// is used.
	EtcdWaitTimeout time.Duration

	// Skip TLS certificate check for etcd connections.
	EtcdInsecure bool

	// If set enables skipper to generate based on ingress resources in kubernetes cluster
	Kubernetes bool

	// If set makes skipper authenticate with the kubernetes API server with service account assigned to the
	// skipper POD.
	// If omitted skipper will rely on kubectl proxy to authenticate with API server
	KubernetesInCluster bool

	// Kubernetes API base URL. Only makes sense if KubernetesInCluster is set to false. If omitted and
	// skipper is not running in-cluster, the default API URL will be used.
	KubernetesURL string

	// KubernetesHealthcheck, when Kubernetes ingress is set, indicates
	// whether an automatic healthcheck route should be generated. The
	// generated route will report healthyness when the Kubernetes API
	// calls are successful. The healthcheck endpoint is accessible from
	// internal IPs, with the path /kube-system/healthz.
	KubernetesHealthcheck bool

	// KubernetesHTTPSRedirect, when Kubernetes ingress is set, indicates
	// whether an automatic redirect route should be generated to redirect
	// HTTP requests to their HTTPS equivalent. The generated route will
	// match requests with the X-Forwarded-Proto and X-Forwarded-Port,
	// expected to be set by the load-balancer.
	KubernetesHTTPSRedirect bool

	// KubernetesIngressClass is a regular expression, that will make
	// skipper load only the ingress resources that that have a matching
	// kubernetes.io/ingress.class annotation. For backwards compatibility,
	// the ingresses without an annotation, or an empty annotation, will
	// be loaded, too.
	KubernetesIngressClass string

	// API endpoint of the Innkeeper service, storing route definitions.
	InnkeeperUrl string

	// Fixed token for innkeeper authentication. (Used mainly in
	// development environments.)
	InnkeeperAuthToken string

	// Filters to be prepended to each route loaded from Innkeeper.
	InnkeeperPreRouteFilters string

	// Filters to be appended to each route loaded from Innkeeper.
	InnkeeperPostRouteFilters string

	// Skip TLS certificate check for Innkeeper connections.
	InnkeeperInsecure bool

	// OAuth2 URL for Innkeeper authentication.
	OAuthUrl string

	// Directory where oauth credentials are stored, with file names:
	// client.json and user.json.
	OAuthCredentialsDir string

	// The whitespace separated list of OAuth2 scopes.
	OAuthScope string

	// File containing static route definitions.
	RoutesFile string

	// Polling timeout of the routing data sources.
	SourcePollTimeout time.Duration

	// Deprecated. See ProxyFlags. When used together with ProxyFlags,
	// the values will be combined with |.
	ProxyOptions proxy.Options

	// Flags controlling the proxy behavior.
	ProxyFlags proxy.Flags

	// Tells the proxy maximum how many idle connections can it keep
	// alive.
	IdleConnectionsPerHost int

	// Defines the time period of how often the idle connections maintained
	// by the proxy are closed.
	CloseIdleConnsPeriod time.Duration

	// Flag indicating to ignore trailing slashes in paths during route
	// lookup.
	IgnoreTrailingSlash bool

	// Priority routes that are matched against the requests before
	// the standard routes from the data clients.
	PriorityRoutes []proxy.PriorityRoute

	// Specifications of custom, user defined predicates.
	CustomPredicates []routing.PredicateSpec

	// Custom data clients to be used together with the default etcd and Innkeeper.
	CustomDataClients []routing.DataClient

	// SuppressRouteUpdateLogs indicates to log only summaries of the routing updates
	// instead of full details of the updated/deleted routes.
	SuppressRouteUpdateLogs bool

	// Dev mode. Currently this flag disables prioritization of the
	// consumer side over the feeding side during the routing updates to
	// populate the updated routes faster.
	DevMode bool

	// Network address for the support endpoints
	SupportListener string

	// Deprecated: Network address for the /metrics endpoint
	MetricsListener string

	// Skipper provides a set of metrics with different keys which are exposed via HTTP in JSON
	// You can customize those key names with your own prefix
	MetricsPrefix string

	// EnableProfile exposes profiling information on /profile of the
	// metrics listener.
	EnableProfile bool

	// Flag that enables reporting of the Go garbage collector statistics exported in debug.GCStats
	EnableDebugGcMetrics bool

	// Flag that enables reporting of the Go runtime statistics exported in runtime and specifically runtime.MemStats
	EnableRuntimeMetrics bool

	// If set, detailed response time metrics will be collected
	// for each route, additionally grouped by status and method.
	EnableServeRouteMetrics bool

	// If set, detailed response time metrics will be collected
	// for each host, additionally grouped by status and method.
	EnableServeHostMetrics bool

	// If set, detailed response time metrics will be collected
	// for each backend host
	EnableBackendHostMetrics bool

	// EnableAllFiltersMetrics enables collecting combined filter
	// metrics per each route. Without the DisableMetricsCompatibilityDefaults,
	// it is enabled by default.
	EnableAllFiltersMetrics bool

	// EnableCombinedResponseMetrics enables collecting response time
	// metrics combined for every route.
	EnableCombinedResponseMetrics bool

	// EnableRouteResponseMetrics enables collecting response time
	// metrics per each route. Without the DisableMetricsCompatibilityDefaults,
	// it is enabled by default.
	EnableRouteResponseMetrics bool

	// EnableRouteBackendErrorsCounters enables counters for backend
	// errors per each route. Without the DisableMetricsCompatibilityDefaults,
	// it is enabled by default.
	EnableRouteBackendErrorsCounters bool

	// EnableRouteStreamingErrorsCounters enables counters for streaming
	// errors per each route. Without the DisableMetricsCompatibilityDefaults,
	// it is enabled by default.
	EnableRouteStreamingErrorsCounters bool

	// EnableRouteBackendMetrics enables backend response time metrics
	// per each route. Without the DisableMetricsCompatibilityDefaults, it is
	// enabled by default.
	EnableRouteBackendMetrics bool

	// When set, makes the histograms use an exponentially decaying sample
	// instead of the default uniform one.
	MetricsUseExpDecaySample bool

	// The following options, for backwards compatibility, are true
	// by default: EnableAllFiltersMetrics, EnableRouteResponseMetrics,
	// EnableRouteBackendErrorsCounters, EnableRouteStreamingErrorsCounters,
	// EnableRouteBackendMetrics. With this compatibility flag, the default
	// for these options can be set to false.
	DisableMetricsCompatibilityDefaults bool

	// Output file for the application log. Default value: /dev/stderr.
	//
	// When /dev/stderr or /dev/stdout is passed in, it will be resolved
	// to os.Stderr or os.Stdout.
	//
	// Warning: passing an arbitrary file will try to open it append
	// on start and use it, or fail on start, but the current
	// implementation doesn't support any more proper handling
	// of temporary failures or log-rolling.
	ApplicationLogOutput string

	// Application log prefix. Default value: "[APP]".
	ApplicationLogPrefix string

	// Output file for the access log. Default value: /dev/stderr.
	//
	// When /dev/stderr or /dev/stdout is passed in, it will be resolved
	// to os.Stderr or os.Stdout.
	//
	// Warning: passing an arbitrary file will try to open for append
	// it on start and use it, or fail on start, but the current
	// implementation doesn't support any more proper handling
	// of temporary failures or log-rolling.
	AccessLogOutput string

	// Disables the access log.
	AccessLogDisabled bool

	// Enables logs in JSON format
	AccessLogJSONEnabled bool

	DebugListener string

	//Path of certificate when using TLS
	CertPathTLS string
	//Path of key when using TLS
	KeyPathTLS string

	// Flush interval for upgraded Proxy connections
	BackendFlushInterval time.Duration

	// Experimental feature to handle protocol Upgrades for Websockets, SPDY, etc.
	ExperimentalUpgrade bool

	// MaxLoopbacks defines the maximum number of loops that the proxy can execute when the routing table
	// contains loop backends (<loopback>).
	MaxLoopbacks int

	// EnableBreakers enables the usage of the breakers in the route definitions without initializing any
	// by default. It is a shortcut for setting the BreakerSettings to:
	//
	// 	[]circuit.BreakerSettings{{Type: BreakerDisabled}}
	//
	EnableBreakers bool

	// BreakerSettings contain global and host specific settings for the circuit breakers.
	BreakerSettings []circuit.BreakerSettings

	// EnableRatelimiters enables the usage of the ratelimiter in the route definitions without initializing any
	// by default. It is a shortcut for setting the RatelimitSettings to:
	//
	// 	[]ratelimit.Settings{{Type: DisableRatelimit}}
	//
	EnableRatelimiters bool

	// RatelimitSettings contain global and host specific settings for the ratelimiters.
	RatelimitSettings []ratelimit.Settings

	// OpenTracing enables opentracing
	OpenTracing []string

	// PluginDir defines the dir to load plugins from
	PluginDir string

	// DefaultHTTPStatus is the HTTP status used when no routes are found
	// for a request.
	DefaultHTTPStatus int
}

Options to start skipper.