xds

package
v0.11.6-merged-prereqs Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 20, 2024 License: Apache-2.0 Imports: 32 Imported by: 8

README

xDS

Background

xDS is the set of discovery services and APIs used by Envoy to discover its dynamic resources.

xDS Server

Gloo Edge is an xDS server. It maintains a snapshot-based, in-memory cache and responds to xDS requests with the resources that are requested.

Snapshot

A snapshot is a versioned group of resources. In Gloo Edge, we rely on an Envoy snapshot, a snapshot of the xDS resources that Gloo serves to Envoy.

Snapshot Cache

A snapshot cache maintains a single versioned snapshot per key. It also responds to open xDS requests.

xDS Callbacks

xDS callbacks are a set of callbacks that are invoked asynchronously during the lifecycle of an xDS request.

Gloo Edge open source does not define any xDS callbacks. However, these callbacks are a type of extension that can be injected at runtime. Gloo Edge Enterprise defines xDS callbacks, and injects them into the Control Plane at runtime.

Server

An xDS server defines a set of handlers for streaming discovery requests.

xDS Services

The xDS server is configured to expose the following discovery services in Gloo Edge:

ListenerDiscoveryService

The ListenerDiscoveryService allows Envoy to discover Listeners at runtime.

RouteDiscoveryService

The RouteDiscoveryService allows Envoy to discover routing configuration for an HttpConnectionManager filter at runtime.

ClusterDiscoveryService

The ClusterDiscoveryService allows Envoy to discover routable destinations at runtime.

EndpointDiscoveryService

The EndpointDiscoveryService allows Envoy to discover members in a cluster at runtime.

AggregatedDiscoveryService

The AggregatedDiscoveryService allows Envoy to discover all resource types over a single stream at runtime.

SoloDiscoveryService

The SoloDiscoveryService is a custom xDS service, used to serve resources of Any type, that is based on Envoy's Aggregated Discovery Service.

In addition to serving configuration for Envoy resources, the Gloo xDS server is also responsible for serving configuration to a number of enterprise extensions (ie ext-auth and rate-limit)

The SoloDiscoveryService is required to serve these extension resources. It is largely based on the Envoy v2 API, and since it is purely an internal API, we do not need to upgrade the API to match the Envoy xDS API. This issue contains additional context around the reason behind this custom discovery service.

xDS Requests

Gloo Edge supports managing configuration for multiple proxies through a single xDS server. To do so, it stores each snapshot in the cache at a key that is unique to that proxy.

To guarantee that proxies initiate requests for the snapshot they want, it is critical that we establish a naming pattern for cache keys. This pattern must be used both by the proxies requesting the resources from the cache, and the controllers that set the resources in the cache.

The naming convention that we follow is "NAMESPACE~NAME"

Proxies identify the cache key that they are interested in by specifying their node.metadata.role to the cache key using the above naming pattern. An example of this can be found in the bootstrap configuration for proxies

xDS Debugging

Debugging xDS behavior can be challenging, below are a few techniques to help:

Control Plane Logging

The Gloo translation loop is responsible for converting a Gloo API Snapshot into an xDS Snapshot. After completing a translation loop, there is a log line indicating what key in the snapshot cache was updated.

xDS Debug Endpoint

Gloo supports running in dev mode, and when that is enabled, xDS Snapshots are exposed via an endpoint.

Useful information

Documentation

Overview

Package server provides an implementation of a streaming xDS server.

Package server provides an implementation of a streaming xDS server.

Index

Constants

View Source 
const (
	// FallbackNodeCacheKey is used to let nodes know they have a bad config
	// we assign a "fix me" snapshot for bad nodes
	FallbackNodeCacheKey = "misconfigured-node"

	// KeyDelimiter is the character used to join segments of a cache key
	KeyDelimiter = "~"

	// RoleKey is the name of the ket in the node.metadata used to store the role
	RoleKey = "role"
)

Variables

This section is empty.

Functions

func NewAdsSnapshotCache added in v1.9.25 

func NewAdsSnapshotCache(ctx context.Context) cache.SnapshotCache

NewAdsSnapshotCache returns a snapshot-based cache, used to serve xDS requests

func NewEndpointsSnapshotFromResources added in v1.6.16 

func NewEndpointsSnapshotFromResources(
	endpoints cache.Resources,
	clusters cache.Resources,
) cache.Snapshot

func NewNodeRoleHasher added in v1.9.25 

func NewNodeRoleHasher() *nodeRoleHasher

func NewSnapshotFromResources 

func NewSnapshotFromResources(
	endpoints cache.Resources,
	clusters cache.Resources,
	routes cache.Resources,
	listeners cache.Resources,
) cache.Snapshot

func OwnerNamespaceNameID added in v1.17.0 

func OwnerNamespaceNameID(owner, namespace, name string) string

OwnerNamespaceNameID returns the string identifier for an Envoy node in a provided namespace. Envoy proxies are assigned their configuration by Gloo based on their Node ID. Therefore, proxies must identify themselves using the same naming convention that we use to persist the Proxy resource in the snapshot cache. The naming convention that we follow is "OWNER~NAMESPACE~NAME"

func SetEdsOnCluster added in v0.13.16 

func SetEdsOnCluster(out *envoy_config_cluster_v3.Cluster, settings *v1.Settings)

SetEdsOnCluster marks an Envoy Cluster to receive its Endpoints from the xDS Server https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/service_discovery#arch-overview-service-discovery-types-eds In Gloo, we support both streaming (gRPC) and polling (REST)

NOTE: REST EDS was introduced as a way of bypassing https://github.com/envoyproxy/envoy/issues/13070 a bug in Envoy that would cause clusters to warm, without endpoints. That bug has since been resolved and gRPC EDS is preferred as the polling solution is more resource intensive and could delay updates by as much as 5 seconds (or whatever the refresh delay is)

func SetupEnvoyXds 

func SetupEnvoyXds(grpcServer *grpc.Server, xdsServer envoyserver.Server, envoyCache envoycache.SnapshotCache)

register xDS methods with GRPC server

func SnapshotCacheKey added in v1.9.25 

func SnapshotCacheKey(proxy *v1.Proxy) string

SnapshotCacheKey returns the key used to identify a Proxy resource in a SnapshotCache

func SnapshotCacheKeys added in v1.9.25 

func SnapshotCacheKeys(proxies v1.ProxyList) []string

SnapshotCacheKeys returns a list with the SnapshotCacheKey for each Proxy

Types

type EnvoyServerV3 added in v1.6.0 

type EnvoyServerV3 interface {
	envoy_service_endpoint_v3.EndpointDiscoveryServiceServer
	envoy_service_cluster_v3.ClusterDiscoveryServiceServer
	envoy_service_route_v3.RouteDiscoveryServiceServer
	envoy_service_listener_v3.ListenerDiscoveryServiceServer
	envoy_service_discovery_v3.AggregatedDiscoveryServiceServer
}

Server is a collection of handlers for streaming discovery requests.

func NewEnvoyServerV3 added in v1.6.0 

func NewEnvoyServerV3(genericServer server.Server) EnvoyServerV3

type EnvoySnapshot 

type EnvoySnapshot struct {
	// Endpoints are items in the EDS V3 response payload.
	Endpoints cache.Resources

	// Clusters are items in the CDS response payload.
	Clusters cache.Resources

	// Routes are items in the RDS response payload.
	Routes cache.Resources

	// Listeners are items in the LDS response payload.
	Listeners cache.Resources
}

Snapshot is an internally consistent snapshot of xDS resources. Consistently is important for the convergence as different resource types from the snapshot may be delivered to the proxy in arbitrary order.

func NewSnapshot 

func NewSnapshot(
	version string,
	endpoints []cache.Resource,
	clusters []cache.Resource,
	routes []cache.Resource,
	listeners []cache.Resource,
) *EnvoySnapshot

NewSnapshot creates a snapshot from response types and a version.

func (*EnvoySnapshot) Clone added in v0.15.0 

func (s *EnvoySnapshot) Clone() cache.Snapshot

func (*EnvoySnapshot) Consistent 

func (s *EnvoySnapshot) Consistent() error

Consistent check verifies that the dependent resources are exactly listed in the snapshot: - all EDS resources are listed by name in CDS resources - all RDS resources are listed by name in LDS resources

Note that clusters and listeners are requested without name references, so Envoy will accept the snapshot list of clusters as-is even if it does not match all references found in xDS.

func (*EnvoySnapshot) Equal added in v1.6.0 

func (this *EnvoySnapshot) Equal(that *EnvoySnapshot) bool

Equal checks is 2 snapshots are equal, important since reflect.DeepEqual no longer works with proto4

func (*EnvoySnapshot) GetResources 

func (s *EnvoySnapshot) GetResources(typ string) cache.Resources

GetResources selects snapshot resources by type.

func (*EnvoySnapshot) MakeConsistent added in v1.9.25 

func (s *EnvoySnapshot) MakeConsistent()

MakeConsistent removes any items that fail to link to parent resources in the snapshot. It will also add placeholder routes for listeners referencing non-existent routes.

type GlooXdsServer added in v1.8.0 

type GlooXdsServer interface {
	discovery_service.SoloDiscoveryServiceServer
}

Server includes handlers for streaming aggregate discovery requests via the SoloDiscoveryService.

func NewGlooXdsServer added in v1.8.0 

func NewGlooXdsServer(genericServer server.Server) GlooXdsServer

type MockXdsCache added in v1.17.0 

type MockXdsCache struct {
	Called bool
	// Snap that is set
	SetSnap envoycache.Snapshot
	// Snap that is returned
	GetSnap envoycache.Snapshot
}

MockXdsCache is a custom implementation for the SnapshotCache interface It was copied from `project/gloo/pkg/syncer/syncer_test_helper.MockXdsCache` It is located here for 2 reasons:

  1. It is now co-located with the relevant xds code
  2. It can be imported by other tests, without introducing import cycles

func (*MockXdsCache) ClearSnapshot added in v1.17.0 

func (*MockXdsCache) ClearSnapshot(node string)

func (*MockXdsCache) CreateWatch added in v1.17.0 

func (*MockXdsCache) CreateWatch(envoycache.Request) (value chan envoycache.Response, cancel func())

func (*MockXdsCache) Fetch added in v1.17.0 

func (*MockXdsCache) Fetch(context.Context, envoycache.Request) (*envoycache.Response, error)

func (*MockXdsCache) GetSnapshot added in v1.17.0 

func (c *MockXdsCache) GetSnapshot(node string) (envoycache.Snapshot, error)

func (*MockXdsCache) GetStatusInfo added in v1.17.0 

func (*MockXdsCache) GetStatusInfo(string) envoycache.StatusInfo

func (*MockXdsCache) GetStatusKeys added in v1.17.0 

func (c *MockXdsCache) GetStatusKeys() []string

func (*MockXdsCache) SetSnapshot added in v1.17.0 

func (c *MockXdsCache) SetSnapshot(node string, snapshot envoycache.Snapshot)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.