proxy

README

proxy

proxy facilitates both a basic reverse proxy and a robust load balancer.

The proxy has support for multiple backends. The load balancing features include multiple policies, health checks, and failovers. If all hosts fail their health check the proxy middleware will fail back to randomly selecting a target and sending packets to it.

Syntax

In its most basic form, a simple reverse proxy uses this syntax:

proxy FROM TO

• FROM is the base domain to match for the request to be proxied.
• TO is the destination endpoint to proxy to.

However, advanced features including load balancing can be utilized with an expanded syntax:

proxy FROM TO... {
    policy random|least_conn|round_robin
    fail_timeout DURATION
    max_fails INTEGER
    health_check PATH:PORT [DURATION]
    except IGNORED_NAMES...
    spray
    protocol [dns [force_tcp]|https_google [bootstrap ADDRESS...]|grpc [insecure|CACERT|KEY CERT|KEY CERT CACERT]]
}

• FROM is the name to match for the request to be proxied.
• TO is the destination endpoint to proxy to. At least one is required, but multiple may be specified. TO may be an IP:Port pair, or may reference a file in resolv.conf format
• policy is the load balancing policy to use; applies only with multiple backends. May be one of random, least_conn, or round_robin. Default is random.
• fail_timeout specifies how long to consider a backend as down after it has failed. While it is down, requests will not be routed to that backend. A backend is "down" if CoreDNS fails to communicate with it. The default value is 10 seconds ("10s").
• max_fails is the number of failures within fail_timeout that are needed before considering a backend to be down. If 0, the backend will never be marked as down. Default is 1.
• health_check will check path (on port) on each backend. If a backend returns a status code of 200-399, then that backend is marked healthy for double the healthcheck duration. If it doesn't, it is marked as unhealthy and no requests are routed to it. If this option is not provided then health checks are disabled. The default duration is 30 seconds ("30s").
• IGNORED_NAMES in except is a space-separated list of domains to exclude from proxying. Requests that match none of these names will be passed through.
• spray when all backends are unhealthy, randomly pick one to send the traffic to. (This is a failsafe.)
• protocol specifies what protocol to use to speak to an upstream, dns (the default) is plain old DNS, and https_google uses https://dns.google.com and speaks a JSON DNS dialect. Note when using this TO will be ignored. The grpc option will talk to a server that has implemented the DnsService. An out-of-tree middleware that implements the server side of this can be found at here.

Policies

There are three load-balancing policies available:

• random (default) - Randomly select a backend
• least_conn - Select the backend with the fewest active connections
• round_robin - Select the backend in round-robin fashion

All polices implement randomly spraying packets to backend hosts when no healthy hosts are available. This is to preeempt the case where the healthchecking (as a mechanism) fails.

Upstream Protocols

Currently protocol supports dns (i.e., standard DNS over UDP/TCP) and https_google (JSON payload over HTTPS). Note that with https_google the entire transport is encrypted. Only you and Google can see your DNS activity.

• dns: uses the standard DNS exchange. You can pass force_tcp to make sure that the proxied connection is performed over TCP, regardless of the inbound request's protocol.

• https_google: bootstrap ADDRESS... is used to (re-)resolve dns.google.com to an address to connect to. This happens every 300s. If not specified the default is used: 8.8.8.8:53/8.8.4.4:53. Note that TO is ignored when https_google is used, as its upstream is defined as dns.google.com.

  Debug queries are enabled by default and currently there is no way to turn them off. When CoreDNS receives a debug query (i.e. the name is prefixed with o-o.debug.) a TXT record with Comment from dns.google.com is added. Note this is not always set.

• grpc: options are used to control how the TLS connection is made to the gRPC server.

  • None - No client authentication is used, and the system CAs are used to verify the server certificate.
  • insecure - TLS is not used, the connection is made in plaintext (not good in production).
  • CACERT - No client authentication is used, and the file CACERT is used to verify the server certificate.
  • KEY CERT - Client authentication is used with the specified key/cert pair. The server certificate is verified with the system CAs.
  • KEY CERT CACERT - Client authentication is used with the specified key/cert pair. The server certificate is verified using the CACERT file.

  An out-of-tree middleware that implements the server side of this can be found at here.

Metrics

If monitoring is enabled (via the prometheus directive) then the following metric is exported:

• coredns_proxy_request_count_total{proto, proxy_proto, from}

Where proxy_proto is the protocol used (dns, grpc, or https_google) and from is FROM specified in the config, proto is the protocol used by the incoming query ("tcp" or "udp").

Examples

Proxy all requests within example.org. to a backend system:

proxy example.org localhost:9005

Load-balance all requests between three backends (using random policy):

proxy . dns1.local:53 dns2.local:1053 dns3.local

Same as above, but round-robin style:

proxy . dns1.local:53 dns2.local:1053 dns3.local {
	policy round_robin
}

With health checks and proxy headers to pass hostname, IP, and scheme upstream:

proxy . dns1.local:53 dns2.local:53 dns3.local:53 {
	policy round_robin
	health_check /health:8080
}

Proxy everything except requests to miek.nl or example.org

proxy . backend:1234 {
	except miek.nl example.org
}

Proxy everything except example.org using the host resolv.conf nameservers:

proxy . /etc/resolv.conf {
	except miek.nl example.org
}

Proxy all requests within example.org to Google's dns.google.com.

proxy example.org 1.2.3.4:53 {
    protocol https_google
}

Proxy everything with HTTPS to dns.google.com, except example.org. Then have another proxy in another stanza that uses plain DNS to resolve names under example.org.

. {
    proxy . 1.2.3.4:53 {
        except example.org
        protocol https_google
    }
}

example.org {
    proxy . 8.8.8.8:53
}

Documentation

Overview

Package proxy is middleware that proxies requests.

Index

• Variables
• func OnStartupMetrics() error
• func RegisterPolicy(name string, policy func() Policy)
• type Exchanger
• type HostPool
• type LeastConn
  • func (r *LeastConn) Select(pool HostPool) *UpstreamHost
• type Options
• type Policy
• type Proxy
  • func NewLookup(hosts []string) Proxy
  • func NewLookupWithOption(hosts []string, opts Options) Proxy
  • func (p Proxy) Forward(state request.Request) (*dns.Msg, error)
  • func (p Proxy) Lookup(state request.Request, name string, typ uint16) (*dns.Msg, error)
  • func (p Proxy) Name() string
  • func (p Proxy) ServeDNS(ctx context.Context, w dns.ResponseWriter, r *dns.Msg) (int, error)
• type Random
  • func (r *Random) Select(pool HostPool) *UpstreamHost
• type RoundRobin
  • func (r *RoundRobin) Select(pool HostPool) *UpstreamHost
• type Spray
  • func (r *Spray) Select(pool HostPool) *UpstreamHost
• type Upstream
  • func NewStaticUpstreams(c *caddyfile.Dispenser) ([]Upstream, error)
• type UpstreamHost
  • func (uh *UpstreamHost) Down() bool
• type UpstreamHostDownFunc

Variables

var (
	RequestDuration = prometheus.NewHistogramVec(prometheus.HistogramOpts{
		Namespace: middleware.Namespace,
		Subsystem: "proxy",
		Name:      "request_duration_milliseconds",
		Buckets:   append(prometheus.DefBuckets, []float64{50, 100, 200, 500, 1000, 2000, 3000, 4000, 5000, 10000}...),
		Help:      "Histogram of the time (in milliseconds) each request took.",
	}, []string{"proto", "proxy_proto", "from"})
)

Metrics the proxy middleware exports.

Functions

func OnStartupMetrics

func OnStartupMetrics() error

OnStartupMetrics sets up the metrics on startup. This is done for all proxy protocols.

func RegisterPolicy

func RegisterPolicy(name string, policy func() Policy)

RegisterPolicy adds a custom policy to the proxy.

Types

type Exchanger

type Exchanger interface {
	Exchange(ctx context.Context, addr string, state request.Request) (*dns.Msg, error)
	Protocol() string

	OnStartup(*Proxy) error
	OnShutdown(*Proxy) error
}

Exchanger is an interface that specifies a type implementing a DNS resolver that can use whatever transport it likes.

type HostPool

type HostPool []*UpstreamHost

HostPool is a collection of UpstreamHosts.

type LeastConn

type LeastConn struct{}

LeastConn is a policy that selects the host with the least connections.

func (*LeastConn) Select

func (r *LeastConn) Select(pool HostPool) *UpstreamHost

Select selects the up host with the least number of connections in the pool. If more than one host has the same least number of connections, one of the hosts is chosen at random.

type Options

type Options struct {
	ForceTCP bool // If true use TCP for upstream no matter what
}

Options define the options understood by dns.Exchange.

type Policy

type Policy interface {
	Select(pool HostPool) *UpstreamHost
}

Policy decides how a host will be selected from a pool. When all hosts are unhealthy, it is assumed the healthchecking failed. In this case each policy will *randomly* return a host from the pool to prevent no traffic to go through at all.

type Proxy

type Proxy struct {
	Next middleware.Handler

	Upstreams *[]Upstream

	// Trace is the Trace middleware, if it is installed
	// This is used by the grpc exchanger to trace through the grpc calls
	Trace middleware.Handler
}

Proxy represents a middleware instance that can proxy requests to another (DNS) server.

func NewLookup

func NewLookup(hosts []string) Proxy

NewLookup create a new proxy with the hosts in host and a Random policy.

func NewLookupWithOption

func NewLookupWithOption(hosts []string, opts Options) Proxy

NewLookupWithOption process creates a simple round robin forward with potentially forced proto for upstream.

func (Proxy) Forward

func (p Proxy) Forward(state request.Request) (*dns.Msg, error)

Forward forward the request in state as-is. Unlike Lookup that adds EDNS0 suffix to the message.

func (Proxy) Lookup

func (p Proxy) Lookup(state request.Request, name string, typ uint16) (*dns.Msg, error)

Lookup will use name and type to forge a new message and will send that upstream. It will set any EDNS0 options correctly so that downstream will be able to process the reply.

func (Proxy) Name

func (p Proxy) Name() string

Name implements the Handler interface.

func (Proxy) ServeDNS

func (p Proxy) ServeDNS(ctx context.Context, w dns.ResponseWriter, r *dns.Msg) (int, error)

ServeDNS satisfies the middleware.Handler interface.

type Random

type Random struct{}

Random is a policy that selects up hosts from a pool at random.

func (*Random) Select

func (r *Random) Select(pool HostPool) *UpstreamHost

Select selects an up host at random from the specified pool.

type RoundRobin

type RoundRobin struct {
	Robin uint32
}

RoundRobin is a policy that selects hosts based on round robin ordering.

func (*RoundRobin) Select

func (r *RoundRobin) Select(pool HostPool) *UpstreamHost

Select selects an up host from the pool using a round robin ordering scheme.

type Spray

type Spray struct{}

Spray is a policy that selects a host from a pool at random. This should be used as a last ditch attempt to get a host when all hosts are reporting unhealthy.

func (*Spray) Select

func (r *Spray) Select(pool HostPool) *UpstreamHost

Select selects an up host at random from the specified pool.

type Upstream

type Upstream interface {
	// The domain name this upstream host should be routed on.
	From() string
	// Selects an upstream host to be routed to.
	Select() *UpstreamHost
	// Checks if subpdomain is not an ignored.
	IsAllowedDomain(string) bool
	// Exchanger returns the exchanger to be used for this upstream.
	Exchanger() Exchanger
	// Stops the upstream from proxying requests to shutdown goroutines cleanly.
	Stop() error
}

Upstream manages a pool of proxy upstream hosts. Select should return a suitable upstream host, or nil if no such hosts are available.

func NewStaticUpstreams

func NewStaticUpstreams(c *caddyfile.Dispenser) ([]Upstream, error)

NewStaticUpstreams parses the configuration input and sets up static upstreams for the proxy middleware.

type UpstreamHost

type UpstreamHost struct {
	Conns             int64  // must be first field to be 64-bit aligned on 32-bit systems
	Name              string // IP address (and port) of this upstream host
	Fails             int32
	FailTimeout       time.Duration
	OkUntil           time.Time
	CheckDown         UpstreamHostDownFunc
	CheckURL          string
	WithoutPathPrefix string
	Checking          bool
	// contains filtered or unexported fields
}

UpstreamHost represents a single proxy upstream

func (*UpstreamHost) Down

func (uh *UpstreamHost) Down() bool

Down checks whether the upstream host is down or not. Down will try to use uh.CheckDown first, and will fall back to some default criteria if necessary.

type UpstreamHostDownFunc

type UpstreamHostDownFunc func(*UpstreamHost) bool

UpstreamHostDownFunc can be used to customize how Down behaves.