proxycfg

Documentation

Overview

Package proxycfg provides a component that monitors local agent state for Connect proxy service registrations and maintains the necessary cache state for those proxies locally. Local cache state keeps pull based proxies (e.g. the built in one) performant even on first request/startup, and allows for push-based proxy APIs (e.g. xDS for Envoy) to be notified of updates to the proxy configuration.

The relationship with other agent components looks like this:

   +------------------------------------------+
   | AGENT                                    |
   |                                          |
   | +--------+  1.  +----------+             |
   | | local  |<-----+ proxycfg |<--------+   |
   | | state  +----->| Manager  |<---+    |   |
   | +--------+  2.  +^---+-----+    |    |   |
   |                5.|   |          |    |   |
   |       +----------+   |  +-------+--+ |4. |
   |       |              +->| proxycfg | |   |
   |       |            3.|  |  State   | |   |
   |       |              |  +----------+ |   |
   |       |              |               |   |
   |       |              |  +----------+ |   |
   |       |              +->| proxycfg +-+   |
   |       |                 |  State   |     |
   |       |                 +----------+     |
   |       |6.                                |
   |  +----v---+                              |
   |  |   xDS  |                              |
   |  | Server |                              |
   |  +--------+                              |
   |                                          |
   +------------------------------------------+

1. Manager watches local state for changes.
2. On local state change manager is notified and iterates through state
   looking for proxy service registrations.
3. For each proxy service registered, the manager maintains a State
   instance, recreating on change, removing when deregistered.
4. State instance copies the parts of the the proxy service registration
   needed to configure proxy, and sets up blocking watches on the local
   agent cache for all remote state needed: root and leaf certs, intentions,
   and service discovery results for the specified upstreams. This ensures
   these results are always in local cache for "pull" based proxies like the
   built-in one.
5. If needed, pull-based proxy config APIs like the xDS server can Watch the
   config for a given proxy service.
6. Watchers get notified every time something changes the current snapshot
   of config for the proxy. That might be changes to the registration,
   certificate rotations, changes to the upstreams required (needing
   different listener config), or changes to the service discovery results
   for any upstream (e.g. new instance of upstream service came up).

Index

• Variables
• func TestCacheWithTypes(t testing.T, types *TestCacheTypes) *cache.Cache
• func TestCerts(t testing.T) (*structs.IndexedCARoots, *structs.IssuedCert)
• func TestGatewayNodesDC2(t testing.T) structs.CheckServiceNodes
• func TestGatewayServiceGroupBarDC1(t testing.T) structs.CheckServiceNodes
• func TestGatewayServiceGroupFooDC1(t testing.T) structs.CheckServiceNodes
• func TestIntentions(t testing.T) *structs.IndexedIntentionMatches
• func TestLeafForCA(t testing.T, ca *structs.CARoot) *structs.IssuedCert
• func TestUpstreamNodes(t testing.T) structs.CheckServiceNodes
• func TestUpstreamNodesAlternate(t testing.T) structs.CheckServiceNodes
• func TestUpstreamNodesDC2(t testing.T) structs.CheckServiceNodes
• type CacheNotifier
• type CancelFunc
• type ConfigSnapshot
  • func TestConfigSnapshot(t testing.T) *ConfigSnapshot
  • func TestConfigSnapshotDiscoveryChain(t testing.T) *ConfigSnapshot
  • func TestConfigSnapshotDiscoveryChainWithEntries(t testing.T, additionalEntries ...structs.ConfigEntry) *ConfigSnapshot
  • func TestConfigSnapshotDiscoveryChainWithFailover(t testing.T) *ConfigSnapshot
  • func TestConfigSnapshotDiscoveryChain_SplitterWithResolverRedirectMultiDC(t testing.T) *ConfigSnapshot
  • func TestConfigSnapshotMeshGateway(t testing.T) *ConfigSnapshot
  • func (s *ConfigSnapshot) Clone() (*ConfigSnapshot, error)
  • func (s *ConfigSnapshot) Valid() bool
• type ControllableCacheType
  • func NewControllableCacheType(t testing.T) *ControllableCacheType
  • func (ct *ControllableCacheType) Fetch(opts cache.FetchOptions, req cache.Request) (cache.FetchResult, error)
  • func (ct *ControllableCacheType) Set(key string, value interface{})
  • func (ct *ControllableCacheType) SupportsBlocking() bool
• type Manager
  • func NewManager(cfg ManagerConfig) (*Manager, error)
  • func (m *Manager) Close() error
  • func (m *Manager) Run() error
  • func (m *Manager) Watch(proxyID string) (<-chan *ConfigSnapshot, CancelFunc)
• type ManagerConfig
• type TestCacheTypes
  • func NewTestCacheTypes(t testing.T) *TestCacheTypes

Variables

var (
	// ErrStopped is returned from Run if the manager instance has already been
	// stopped.
	ErrStopped = errors.New("manager stopped")

	// ErrStarted is returned from Run if the manager instance has already run.
	ErrStarted = errors.New("manager was already run")
)

Functions

func TestCacheWithTypes

func TestCacheWithTypes(t testing.T, types *TestCacheTypes) *cache.Cache

TestCacheWithTypes registers ControllableCacheTypes for all types that proxycfg will watch suitable for testing a proxycfg.State or Manager.

func TestCerts

func TestCerts(t testing.T) (*structs.IndexedCARoots, *structs.IssuedCert)

TestCerts generates a CA and Leaf suitable for returning as mock CA root/leaf cache requests.

func TestGatewayNodesDC2

func TestGatewayNodesDC2(t testing.T) structs.CheckServiceNodes

func TestGatewayServiceGroupBarDC1

func TestGatewayServiceGroupBarDC1(t testing.T) structs.CheckServiceNodes

func TestGatewayServiceGroupFooDC1

func TestGatewayServiceGroupFooDC1(t testing.T) structs.CheckServiceNodes

func TestIntentions

func TestIntentions(t testing.T) *structs.IndexedIntentionMatches

TestIntentions returns a sample intentions match result useful to mocking service discovery cache results.

func TestLeafForCA

func TestLeafForCA(t testing.T, ca *structs.CARoot) *structs.IssuedCert

TestLeafForCA generates new Leaf suitable for returning as mock CA leaf cache response, signed by an existing CA.

func TestUpstreamNodes

func TestUpstreamNodes(t testing.T) structs.CheckServiceNodes

TestUpstreamNodes returns a sample service discovery result useful to mocking service discovery cache results.

func TestUpstreamNodesAlternate

func TestUpstreamNodesAlternate(t testing.T) structs.CheckServiceNodes

func TestUpstreamNodesDC2

func TestUpstreamNodesDC2(t testing.T) structs.CheckServiceNodes

Types

type CacheNotifier

type CacheNotifier interface {
	Notify(ctx context.Context, t string, r cache.Request,
		correlationID string, ch chan<- cache.UpdateEvent) error
}

type CancelFunc

type CancelFunc func()

CancelFunc is a type for a returned function that can be called to cancel a watch.

type ConfigSnapshot

type ConfigSnapshot struct {
	Kind            structs.ServiceKind
	Service         string
	ProxyID         string
	Address         string
	Port            int
	TaggedAddresses map[string]structs.ServiceAddress
	Proxy           structs.ConnectProxyConfig
	Datacenter      string
	Roots           *structs.IndexedCARoots

	// connect-proxy specific
	ConnectProxy configSnapshotConnectProxy

	// mesh-gateway specific
	MeshGateway configSnapshotMeshGateway
}

ConfigSnapshot captures all the resulting config needed for a proxy instance. It is meant to be point-in-time coherent and is used to deliver the current config state to observers who need it to be pushed in (e.g. XDS server).

func TestConfigSnapshot

func TestConfigSnapshot(t testing.T) *ConfigSnapshot

TestConfigSnapshot returns a fully populated snapshot

func TestConfigSnapshotDiscoveryChain

func TestConfigSnapshotDiscoveryChain(t testing.T) *ConfigSnapshot

TestConfigSnapshotDiscoveryChain returns a fully populated snapshot using a discovery chain

func TestConfigSnapshotDiscoveryChainWithEntries

func TestConfigSnapshotDiscoveryChainWithEntries(t testing.T, additionalEntries ...structs.ConfigEntry) *ConfigSnapshot

func TestConfigSnapshotDiscoveryChainWithFailover

func TestConfigSnapshotDiscoveryChainWithFailover(t testing.T) *ConfigSnapshot

func TestConfigSnapshotDiscoveryChain_SplitterWithResolverRedirectMultiDC

func TestConfigSnapshotDiscoveryChain_SplitterWithResolverRedirectMultiDC(t testing.T) *ConfigSnapshot

func TestConfigSnapshotMeshGateway

func TestConfigSnapshotMeshGateway(t testing.T) *ConfigSnapshot

func (*ConfigSnapshot) Clone

func (s *ConfigSnapshot) Clone() (*ConfigSnapshot, error)

Clone makes a deep copy of the snapshot we can send to other goroutines without worrying that they will racily read or mutate shared maps etc.

func (*ConfigSnapshot) Valid

func (s *ConfigSnapshot) Valid() bool

Valid returns whether or not the snapshot has all required fields filled yet.

type ControllableCacheType

type ControllableCacheType struct {
	// contains filtered or unexported fields
}

ControllableCacheType is a cache.Type that simulates a typical blocking RPC but lets us control the responses and when they are delivered easily.

func NewControllableCacheType

func NewControllableCacheType(t testing.T) *ControllableCacheType

NewControllableCacheType returns a cache.Type that can be controlled for testing.

func (*ControllableCacheType) Fetch

func (ct *ControllableCacheType) Fetch(opts cache.FetchOptions, req cache.Request) (cache.FetchResult, error)

Fetch implements cache.Type. It simulates blocking or non-blocking queries.

func (*ControllableCacheType) Set

func (ct *ControllableCacheType) Set(key string, value interface{})

Set sets the response value to be returned from subsequent cache gets for the type.

func (*ControllableCacheType) SupportsBlocking

func (ct *ControllableCacheType) SupportsBlocking() bool

SupportsBlocking implements cache.Type

type Manager

type Manager struct {
	ManagerConfig
	// contains filtered or unexported fields
}

Manager is a component that integrates into the agent and manages Connect proxy configuration state. This should not be confused with the deprecated "managed proxy" concept where the agent supervises the actual proxy process. proxycfg.Manager is oblivious to the distinction and manages state for any service registered with Kind == connect-proxy.

The Manager ensures that any Connect proxy registered on the agent has all the state it needs cached locally via the agent cache. State includes certificates, intentions, and service discovery results for any declared upstreams. See package docs for more detail.

func NewManager

func NewManager(cfg ManagerConfig) (*Manager, error)

NewManager constructs a manager from the provided agent cache.

func (*Manager) Close

func (m *Manager) Close() error

Close removes all state and stops all running goroutines.

func (*Manager) Run

func (m *Manager) Run() error

Run is the long-running method that handles state syncing. It should be run in it's own goroutine and will continue until a fatal error is hit or Close is called. Run will return an error if it is called more than once, or called after Close.

func (*Manager) Watch

func (m *Manager) Watch(proxyID string) (<-chan *ConfigSnapshot, CancelFunc)

Watch registers a watch on a proxy. It might not exist yet in which case this will not fail, but no updates will be delivered until the proxy is registered. If there is already a valid snapshot in memory, it will be delivered immediately.

type ManagerConfig

type ManagerConfig struct {
	// Cache is the agent's cache instance that can be used to retrieve, store and
	// monitor state for the proxies.
	Cache *cache.Cache
	// state is the agent's local state to be watched for new proxy registrations.
	State *local.State
	// source describes the current agent's identity, it's used directly for
	// prepared query discovery but also indirectly as a way to pass current
	// Datacenter name into other request types that need it. This is sufficient
	// for now and cleaner than passing the entire RuntimeConfig.
	Source *structs.QuerySource
	// logger is the agent's logger to be used for logging logs.
	Logger *log.Logger
}

ManagerConfig holds the required external dependencies for a Manager instance. All fields must be set to something valid or the manager will panic. The ManagerConfig is passed by value to NewManager so the passed value can be mutated safely.

type TestCacheTypes

type TestCacheTypes struct {
	// contains filtered or unexported fields
}

TestCacheTypes encapsulates all the different cache types proxycfg.State will watch/request for controlling one during testing.

func NewTestCacheTypes

func NewTestCacheTypes(t testing.T) *TestCacheTypes

NewTestCacheTypes creates a set of ControllableCacheTypes for all types that proxycfg will watch suitable for testing a proxycfg.State or Manager.