kubernetes

Documentation

Overview

Package kubernetes implements Kubernetes Ingress support for Skipper.

See: http://kubernetes.io/docs/user-guide/ingress/

The package provides a Skipper DataClient implementation that can be used to access the Kubernetes API for ingress resources and generate routes based on them. The client polls for the ingress settings, and there is no need for a separate controller. On the other hand, it doesn't provide a full Ingress solution alone, because it doesn't do any load balancer configuration or DNS updates. For a full Ingress solution, it is possible to use Skipper together with Kube-ingress-aws-controller, which targets AWS and takes care of the load balancer setup for Kubernetes Ingress.

See: https://github.com/zalando-incubator/kube-ingress-aws-controller

Both Kube-ingress-aws-controller and Skipper Kubernetes are part of the larger project, Kubernetes On AWS:

https://github.com/zalando-incubator/kubernetes-on-aws/

Ingress shutdown by healthcheck

The Kubernetes ingress client catches TERM signals when the ProvideHealthcheck option is enabled, and reports failing healthcheck after the signal was received. This means that, when the Ingress client is responsible for the healthcheck of the cluster, and the Skipper process receives the TERM signal, it won't exit by itself immediately, but will start reporting failures on healthcheck requests. Until it gets killed by the kubelet, Skipper keeps serving the requests in this case.

Example - Ingress

A basic ingress specification:

apiVersion: extensions/v1beta1
kind: Ingress
Metadata:
  name: app
spec:
  rules:
  - host: app-default.example.org
    http:
      paths:
      - backend:
          serviceName: app-svc
          servicePort: 80

Example - Ingress with ratelimiting

The example shows 50 calls per minute are allowed to each skipper instance for the given ingress.

apiVersion: extensions/v1beta1
kind: Ingress
Metadata:
  annotations:
    zalando.org/ratelimit: ratelimit(50, "1m")
  name: app
spec:
  rules:
  - host: app-default.example.org
    http:
      paths:
      - backend:
          serviceName: app-svc
          servicePort: 80

Example - Ingress with client based ratelimiting

The example shows 3 calls per minute per client, based on X-Forwarded-For header or IP incase there is no X-Forwarded-For header set, are allowed to each skipper instance for the given ingress.

apiVersion: extensions/v1beta1
kind: Ingress
Metadata:
  annotations:
    zalando.org/ratelimit: localRatelimit(3, "1m")
  name: app
spec:
  rules:
  - host: app-default.example.org
    http:
      paths:
      - backend:
          serviceName: app-svc
          servicePort: 80

The example shows 500 calls per hour per client, based on Authorization header set, are allowed to each skipper instance for the given ingress.

apiVersion: extensions/v1beta1
kind: Ingress
Metadata:
  annotations:
    zalando.org/ratelimit: localRatelimit(500, "1h", "auth")
  name: app
spec:
  rules:
  - host: app-default.example.org
    http:
      paths:
      - backend:
          serviceName: app-svc
          servicePort: 80

Example - Ingress with custom skipper filter configuration

The example shows the use of 2 filters from skipper for the implicitly defined route in ingress.

apiVersion: extensions/v1beta1
kind: Ingress
Metadata:
  annotations:
    zalando.org/skipper-filter: localRatelimit(50, "10m") -> requestCookie("test-session", "abc")
  name: app
spec:
  rules:
  - host: app-default.example.org
    http:
      paths:
      - backend:
          serviceName: app-svc
          servicePort: 80

Example - Ingress with custom skipper Predicate configuration

The example shows the use of a skipper predicates for the implicitly defined route in ingress.

apiVersion: extensions/v1beta1
kind: Ingress
Metadata:
  annotations:
    zalando.org/skipper-predicate: QueryParam("query", "^example$")
  name: app
spec:
  rules:
  - host: app-default.example.org
    http:
      paths:
      - backend:
          serviceName: app-svc
          servicePort: 80

Example - Ingress with custom skipper Routes configuration

The example shows the use of custom skipper routes which be additional to the routes generated for the ingress.

apiVersion: extensions/v1beta1
kind: Ingress
Metadata:
  annotations:
    zalando.org/skipper-routes: |
      Method("OPTIONS") ->
      setResponseHeader("Access-Control-Allow-Origin", "*") ->
      setResponseHeader("Access-Control-Allow-Methods", "GET, OPTIONS") ->
      setResponseHeader("Access-Control-Allow-Headers", "Authorization") ->
      status(200) -> <shunt>
  name: app
spec:
  rules:
  - host: app-default.example.org
    http:
      paths:
      - backend:
          serviceName: app-svc
          servicePort: 80

Example - Ingress with shadow traffic

This will send production traffic to app-default.example.org and copies incoming requests to https://app.shadow.example.org, but drops responses from shadow URL. This is helpful to test your next generation software with production workload. See also https://godoc.org/github.com/zalando/skipper/filters/tee for details.

apiVersion: extensions/v1beta1
kind: Ingress
Metadata:
  annotations:
    zalando.org/skipper-filter: tee("https://app.shadow.example.org")
  name: app
spec:
  rules:
  - host: app-default.example.org
    http:
      paths:
      - backend:
          serviceName: app-svc
          servicePort: 80

Index

• Constants
• type Client
  • func New(o Options) (*Client, error)
  • func (c *Client) Close()
  • func (c *Client) GetEndpointAddresses(ns, name string, port int) []string
  • func (c *Client) LoadAll() ([]*eskip.Route, error)
  • func (c *Client) LoadUpdate() ([]*eskip.Route, []string, error)
• type ClusterResource
• type ClusterResourceList
• type Options
• type PathMode
  • func ParsePathMode(s string) (PathMode, error)
  • func (m PathMode) String() string

Constants

const (
	IngressesClusterURI        = "/apis/extensions/v1beta1/ingresses"
	IngressesV1ClusterURI      = "/apis/networking.k8s.io/v1/ingresses"
	ZalandoResourcesClusterURI = "/apis/zalando.org/v1"
	RouteGroupsName            = "routegroups"

	ServicesClusterURI  = "/api/v1/services"
	EndpointsClusterURI = "/api/v1/endpoints"
	SecretsClusterURI   = "/api/v1/secrets"

	IngressesNamespaceFmt   = "/apis/extensions/v1beta1/namespaces/%s/ingresses"
	IngressesV1NamespaceFmt = "/apis/networking.k8s.io/v1/namespaces/%s/ingresses"

	ServicesNamespaceFmt  = "/api/v1/namespaces/%s/services"
	EndpointsNamespaceFmt = "/api/v1/namespaces/%s/endpoints"
	SecretsNamespaceFmt   = "/api/v1/namespaces/%s/secrets"
)

const RouteGroupsNotInstalledMessage = `` /* 129-byte string literal not displayed */

Types

type Client

type Client struct {
	ClusterClient *clusterClient
	// contains filtered or unexported fields
}

Client is a Skipper DataClient implementation used to create routes based on Kubernetes Ingress settings.

func New

func New(o Options) (*Client, error)

New creates and initializes a Kubernetes DataClient.

func (*Client) Close

func (c *Client) Close()

func (*Client) GetEndpointAddresses

func (c *Client) GetEndpointAddresses(ns, name string, port int) []string

func (*Client) LoadAll

func (c *Client) LoadAll() ([]*eskip.Route, error)

func (*Client) LoadUpdate

func (c *Client) LoadUpdate() ([]*eskip.Route, []string, error)

LoadUpdate returns all known eskip.Route, a list of route IDs scheduled for delete and an error.

TODO: implement a force reset after some time.

type ClusterResource

type ClusterResource struct {
	Name string `json:"name"`
}

type ClusterResourceList

type ClusterResourceList struct {

	// Items, aka "resources".
	Items []*ClusterResource `json:"resources"`
}

type Options

type Options struct {
	// KubernetesInCluster defines if skipper is deployed and running in the kubernetes cluster
	// this would make authentication with API server happen through the service account, rather than
	// running along side kubectl proxy
	KubernetesInCluster bool

	// KubernetesURL is used as the base URL for Kubernetes API requests. Defaults to http://localhost:8001.
	// (TBD: support in-cluster operation by taking the address and certificate from the standard Kubernetes
	// environment variables.)
	KubernetesURL string

	// KubernetesNamespace is used to switch between finding ingresses in the cluster-scope or limit
	// the ingresses to only those in the specified namespace. Defaults to "" which means monitor ingresses
	// in the cluster-scope.
	KubernetesNamespace string

	// *DEPRECATED* KubernetesIngressV1 is used to switch between v1beta1 and v1. Kubernetes version 1.22 stopped support
	// for v1beta1, so we have to provide a migration path and this will someday become the default.
	KubernetesIngressV1 bool

	// *DEPRECATED* KubernetesEnableEastWest if set adds automatically routes
	// with "%s.%s.skipper.cluster.local" domain pattern
	KubernetesEnableEastWest bool

	// ProvideHealthcheck, when set, tells the data client to append a healthcheck route to the ingress
	// routes in case of successfully receiving the ingress items from the API (even if individual ingress
	// items may be invalid), or a failing healthcheck route when the API communication fails. The
	// healthcheck endpoint can be accessed from internal IPs on any hostname, with the path
	// /kube-system/healthz.
	//
	// When used in a custom configuration, the current filter registry needs to include the status()
	// filter, and the available predicates need to include the Source() predicate.
	ProvideHealthcheck bool

	// ProvideHTTPSRedirect, when set, tells the data client to append an HTTPS redirect route to the
	// ingress routes. This route will detect the X-Forwarded-Proto=http and respond with a 301 message
	// to the HTTPS equivalent of the same request (using the redirectTo(301, "https:") filter). The
	// X-Forwarded-Proto and X-Forwarded-Port is expected to be set by the load balancer.
	//
	// (See also https://github.com/zalando-incubator/kube-ingress-aws-controller as part of the
	// https://github.com/zalando-incubator/kubernetes-on-aws project.)
	ProvideHTTPSRedirect bool

	// HTTPSRedirectCode, when set defines which redirect code to use for redirecting from HTTP to HTTPS.
	// By default, 308 StatusPermanentRedirect is used.
	HTTPSRedirectCode int

	// IngressClass is a regular expression to filter only those ingresses that match. If an ingress does
	// not have a class annotation or the annotation is an empty string, skipper will load it. The default
	// value for the ingress class is 'skipper'.
	//
	// For further information see:
	//		https://github.com/nginxinc/kubernetes-ingress/tree/master/examples/multiple-ingress-controllers
	IngressClass string

	// RouteGroupClass is a regular expression to filter only those RouteGroups that match. If a RouteGroup
	// does not have the required annotation (zalando.org/routegroup.class) or the annotation is an empty string,
	// skipper will load it. The default value for the RouteGroup class is 'skipper'.
	RouteGroupClass string

	// ReverseSourcePredicate set to true will do the Source IP
	// whitelisting for the heartbeat endpoint correctly in AWS.
	// Amazon's ALB writes the client IP to the last item of the
	// string list of the X-Forwarded-For header, in this case you
	// want to set this to true.
	ReverseSourcePredicate bool

	// Noop, WIP.
	ForceFullUpdatePeriod time.Duration

	// WhitelistedHealthcheckCIDR to be appended to the default iprange
	WhitelistedHealthCheckCIDR []string

	// PathMode controls the default interpretation of ingress paths in cases when the ingress doesn't
	// specify it with an annotation.
	PathMode PathMode

	// *DEPRECATED *KubernetesEastWestDomain sets the DNS domain to be
	// used for east west traffic, defaults to "skipper.cluster.local"
	KubernetesEastWestDomain string

	// KubernetesEastWestRangeDomains set the the cluster internal domains for
	// east west traffic. Identified routes to such domains will include
	// the KubernetesEastWestRangePredicates.
	KubernetesEastWestRangeDomains []string

	// KubernetesEastWestRangePredicates set the Predicates that will be
	// appended to routes identified as to KubernetesEastWestRangeDomains.
	KubernetesEastWestRangePredicates []*eskip.Predicate

	// DefaultFiltersDir enables default filters mechanism and sets the location of the default filters.
	// The provided filters are then applied to all routes.
	DefaultFiltersDir string

	// OriginMarker is *deprecated* and not used anymore. It will be deleted in v1.
	OriginMarker bool

	// If the OpenTracing tag containing RouteGroup backend name
	// (using tracingTag filter) should be added to all routes
	BackendNameTracingTag bool

	// OnlyAllowedExternalNames will enable validation of ingress external names and route groups network
	// backend addresses, explicit LB endpoints validation agains the list of patterns in
	// AllowedExternalNames.
	OnlyAllowedExternalNames bool

	// AllowedExternalNames contains regexp patterns of those domain names that are allowed to be
	// used with external name services (type=ExternalName).
	AllowedExternalNames []*regexp.Regexp

	CertificateRegistry *certregistry.CertRegistry
}

Options is used to initialize the Kubernetes DataClient.

type PathMode

type PathMode int

PathMode values are used to control the ingress path interpretation. The path mode can be set globally for all ingress paths, and it can be overruled by the individual ingress rules using the zalando.org/skipper-ingress-path-mode annotation. When path mode is not set, the Kubernetes ingress specification is used, accepting regular expressions with a mandatory leading "/", automatically prepended by the "^" control character.

When PathPrefix is used, the path matching becomes deterministic when a request could match more than one ingress routes otherwise.

const (
	// KubernetesIngressMode is the default path mode. Expects regular expressions
	// with a mandatory leading "/". The expressions are automatically prepended by
	// the "^" control character.
	KubernetesIngressMode PathMode = iota

	// PathRegexp is like KubernetesIngressMode but is not prepended by the "^"
	// control character.
	PathRegexp

	// PathPrefix is like the PathSubtree predicate. E.g. "/foo/bar" will match
	// "/foo/bar" or "/foo/bar/baz", but won't match "/foo/barooz".
	//
	// In this mode, when a Path or a PathSubtree predicate is set in an annotation,
	// the value from the annotation has precedence over the standard ingress path.
	PathPrefix
)

func ParsePathMode

func ParsePathMode(s string) (PathMode, error)

ParsePathMode parses the string representations of the different path modes.

func (PathMode) String

func (m PathMode) String() string

String returns the string representation of the path mode, the same values that are used in the path mode annotation.