ambex

package
v2.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 14, 2021 License: Apache-2.0 Imports: 81 Imported by: 0

README

Ambex: Ambassador Experimental^H^H^H^H^H^H^H^H^H^H^H^H ADS service

Ambassador prior to v0.50.0 worked by writing out Envoy v1 JSON configuration files, then triggering a hot restart of Envoy. This works, but it has some unpleasant limitations:

  • Restarts take awhile, and as a result you can't change the configuration very quickly.
  • Restarts can drop connections (cf Envoy #2776; more on this later).
  • Envoy itself is deprecating the v1 configuration, and will only support v2 in a bit.

To get around these limitations, and generally go for a better experience all 'round, we want to switch to the so-called xDS model, in which Envoy's configuration is supplied by its various "Discovery Services": e.g. the CDS is the Cluster Discovery Service; the EDS is the Endpoint Discovery Service. For Ambassador, the Aggregated Discovery Service or ADS is the one we want to use -- basically, it brings the other services together under one aegis and lets you just tell Envoy "get everything dynamically."

However, the whole ADS thing is a bit of a pain:

  • Envoy makes a bidirectional gRPC stream to the ADS server.
  • The ADS then makes gRPC calls to the Envoy to feed the Envoy configuration elements, but:
  • The ADS has to carefully order things such that the configuration elements match what Envoy expects for consistency.

Rather than do all that logic by hand, we'll use the Envoy go-control-plane[^1] for the heavy lifting. This is also something of a pain, given that it's not well documented, but here's the deal:

  • The root of the world is a SnapshotCache:

    • import github.com/datawire/ambassador/pkg/envoy-control-plane/cache, then refer to cache.SnapshotCache.
    • A collection of internally consistent configuration objects is a Snapshot (cache.Snapshot).
    • Snapshots are collected in the SnapshotCache.
    • A given SnapshotCache can hold configurations for multiple Envoys, identified by the Envoy nodeID, which must be configured for the Envoy.

  • The SnapshotCache can only hold go-control-plane configuration objects, so you have to build these up to hand to the SnapshotCache.

  • The gRPC stuff is handled by a Server:

    • import github.com/datawire/ambassador/pkg/envoy-control-plane/server, then refer to server.Server.
    • Our runManagementServer function (largely ripped off from the go-control-plane tests) gets this running. It takes a SnapshotCache (cleverly called config for no reason I (Flynn) understand) and a standard Go gRPCServer as arguments.
    • ALL the gRPC madness is handled by the Server, with the assistance of the methods in its callback object.

  • Once the Server is running, Envoy can open a gRPC stream to it.

    • On connection, Envoy will get handed the most recent Snapshot that the Server's SnapshotCache knows about.
    • Whenever a newer Snapshot is added to the SnapshotCache, that Snapshot will get sent to the Envoy.

  • We manage the SnapshotCache by loading Envoy configuration files on disk:

    • it ignores files that start with a . (hidden files)
    • it interprets *.json files as JSON-encoded protobuf
    • it interprets *.pb files as text-encoded protobuf
    • all other files are ignored As for when it loads those files:
    • By default when we get a SIGHUP we reload the configuration.
    • When passed the --watch argument we reload whenever any file in the directory changes. Be careful about updating files atomically if you use this!

[^1]: The Envoy go-control-plane usually refers to github.com/envoyproxy/go-control-plane, but we've "forked" it as github.com/datawire/ambassador/pkg/envoy-control-plane in order to build it against the protobufs for our patched Envoy.

Running Ambex

You'll need the Go toolchain, and will want to have a functioning envoy.

Then, you can run the ambex CLI using busyambassador:

go run github.com/datawire/ambassador/cmd/busyambassador ambex ARGs...

If you're on a platform other than GNU/Linux, in order to have a functioning envoy, you may want to run all of this in the builder shell: make shell.

Try it out

You'll want to run both ambex and an instance of envoy with a boostrap config pointing at that `ambex.

  1. First, start the ambex:

    go run github.com/datawire/ambassador/cmd/busyambassador ambex ./example/ambex/

    or

    go run github.com/datawire/ambassador/cmd/busyambassador ambex --watch ./example/ambex/

  2. Second, in another shell, start the envoy:

    envoy -l debug -c ./example/envoy/bootstrap-ads.yaml

You should now be able to run some curls in (yet) another shell:

$ curl localhost:8080/hello
Hello ADS!!!

$ curl localhost:8080/get
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Connection": "close",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.54.0",
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000"
  },
  "origin": "72.74.69.156",
  "url": "http://httpbin.org/get"
}

Edit and/or add more files to the ./example/ambex/ directory in order to play with more configurations and see them reload instantaneously (if you used the --watch flag), or when-triggered (if you didn't use the --watch flag; trigger a relead by signaling the process with killall -HUP ambex).

Clean up

Kill Ambex and Envoy with Ctrl-C.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Decode 

func Decode(ctx context.Context, name string) (proto.Message, error)

func GetAmbassadorDrainTime 

func GetAmbassadorDrainTime(ctx context.Context) time.Duration

The GetAmbassadorDrainTime function retuns the AMBASSADOR_DRAIN_TIME env var as a time.Duration

func JoinEdsClusters 

func JoinEdsClusters(ctx context.Context, clusters []ecp_cache_types.Resource, edsEndpoints map[string]*apiv2.ClusterLoadAssignment) (endpoints []ecp_cache_types.Resource)

JoinEdsClusters will perform an outer join operation between the eds clusters in the supplied clusterlist and the eds endpoint data in the supplied map. It will return a slice of ClusterLoadAssignments (cast to []ecp_cache_types.Resource) with endpoint data for all the eds clusters in the supplied list. If there is no map entry for a given cluster, an empty ClusterLoadAssignment will be synthesized. The result is a set of endpoints that are consistent (by the go-control-plane's definition of consistent) with the input clusters.

func JoinEdsClustersV3 

func JoinEdsClustersV3(ctx context.Context, clusters []ecp_cache_types.Resource, edsEndpoints map[string]*apiv3_endpoint.ClusterLoadAssignment) (endpoints []ecp_cache_types.Resource)

JoinEdsClustersV3 will perform an outer join operation between the eds clusters in the supplied clusterlist and the eds endpoint data in the supplied map. It will return a slice of ClusterLoadAssignments (cast to []ecp_cache_types.Resource) with endpoint data for all the eds clusters in the supplied list. If there is no map entry for a given cluster, an empty ClusterLoadAssignment will be synthesized. The result is a set of endpoints that are consistent (by the go-control-plane's definition of consistent) with the input clusters.

func ListenerToRdsListener 

func ListenerToRdsListener(lnr *apiv2.Listener) (*apiv2.Listener, []*apiv2.RouteConfiguration, error)

ListenerToRdsListener will take a listener definition and extract any inline RouteConfigurations replacing them with a reference to an RDS supplied route configuration. It does not modify the supplied listener, any configuration included in the result is copied from the input.

If the input listener does not match the expected form it is simply copied, i.e. it is the identity transform for any inputs not matching the expected form.

Example Input (that will get transformed in a non-identity way):

  • a listener configured with an http connection manager

  • that specifies an http router

  • that supplies its RouteConfiguration inline via the route_config field

    { "name": "...", ..., "filter_chains": [ { "filter_chain_match": {...}, "filters": [ { "name": "envoy.filters.network.http_connection_manager", "typed_config": { "@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager", "http_filters": [...], "route_config": { "virtual_hosts": [ { "name": "ambassador-listener-8443-*", "domains": ["*"], "routes": [...], } ] } } } ] } ] }

Example Output:

  • a duplicate listener that defines the "rds" field instead of the "route_config" field

  • and a list of route configurations

  • with route_config_name supplied in such a way as to correlate the two together

    lnr, routes, err := ListenerToRdsListener(...)

    lnr = { "name": "...", ..., "filter_chains": [ { "filter_chain_match": {...}, "filters": [ { "name": "envoy.filters.network.http_connection_manager", "typed_config": { "@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager", "http_filters": [...], "rds": { "config_source": { "ads": {} }, "route_config_name": "ambassador-listener-8443-routeconfig-0" } } } ] } ] }

    routes = [ { "name": "ambassador-listener-8443-routeconfig-0", "virtual_hosts": [ { "name": "ambassador-listener-8443-*", "domains": ["*"], "routes": [...], } ] } ]

func Main 

func Main(ctx context.Context, Version string, rawArgs ...string) error

NOTE WELL: this Main() does NOT RUN from entrypoint! This one is only relevant if you explicitly run Ambex by hand.

func Main2 

func Main2(
	ctx context.Context,
	Version string,
	getUsage MemoryGetter,
	fastpathCh <-chan *FastpathSnapshot,
	rawArgs ...string,
) error

func NewV2ExpandedSnapshot 

func NewV2ExpandedSnapshot(v2snap *ecp_v2_cache.Snapshot) v2ExpandedSnapshot

func NewV3ExpandedSnapshot 

func NewV3ExpandedSnapshot(v3snap *ecp_v3_cache.Snapshot) v3ExpandedSnapshot

func Updater 

func Updater(ctx context.Context, updates <-chan Update, getUsage MemoryGetter) error

The Updator function will run forever (or until the ctx is canceled) and look for updates on the incoming channel. If memory usage is constrained as reported by the getUsage function, updates will be rate limited to guarantee that there are only so many stale configs in memory at a time. The function assumes updates are cumulative and it will drop old queued updates if a new update arrives.

func V3ListenerToRdsListener 

func V3ListenerToRdsListener(lnr *apiv3_listener.Listener) (*apiv3_listener.Listener, []*apiv3_route.RouteConfiguration, error)

V3ListenerToRdsListener is the v3 variety of ListnerToRdsListener

Types

type Args 

type Args struct {
	// contains filtered or unexported fields
}

type Endpoint 

type Endpoint struct {
	ClusterName string
	Ip          string
	Port        uint32
	Protocol    string
}

Endpoint contains the subset of fields we bother to expose.

func (*Endpoint) ToLbEndpoint_v2 

func (e *Endpoint) ToLbEndpoint_v2() *v2endpoint.LbEndpoint

ToLBEndpoint_v2 translates to envoy v2 frinedly form of the Endpoint data.

func (*Endpoint) ToLbEndpoint_v3 

func (e *Endpoint) ToLbEndpoint_v3() *v3endpoint.LbEndpoint

ToLBEndpoint_v3 translates to envoy v3 frinedly form of the Endpoint data.

type Endpoints 

type Endpoints struct {
	Entries map[string][]*Endpoint
}

The Endpoints struct is how Endpoint data gets communicated to ambex. This is a bit simpler than the envoy endpoint data structures, and also provides us a layer of indirection to buffer us from changes in envoy configuration, e.g. we can switch from v2 to v3 endpoint data, or add v3 endpoint data fairly easily with this layer of indirection.

func (*Endpoints) RoutesString 

func (e *Endpoints) RoutesString() string

func (*Endpoints) ToMap_v2 

func (e *Endpoints) ToMap_v2() map[string]*v2.ClusterLoadAssignment

ToMap_v2 produces a map with the envoy v2 friendly forms of all the endpoint data.

func (*Endpoints) ToMap_v3 

func (e *Endpoints) ToMap_v3() map[string]*v3endpointconfig.ClusterLoadAssignment

ToMap_v3 produces a map with the envoy v3 friendly forms of all the endpoint data.

type FastpathSnapshot 

type FastpathSnapshot struct {
	Snapshot  *ecp_v2_cache.Snapshot
	Endpoints *Endpoints
}

FastpathSnapshot holds envoy configuration that bypasses python.

type HasherV2 

type HasherV2 struct {
}

Hasher returns node ID as an ID

func (HasherV2) ID 

func (h HasherV2) ID(node *v2core.Node) string

ID function

type HasherV3 

type HasherV3 struct {
}

Hasher returns node ID as an ID

func (HasherV3) ID 

func (h HasherV3) ID(node *v3core.Node) string

ID function

type MemoryGetter 

type MemoryGetter func() int

Function type for fetching memory usage as a percentage.

type Update 

type Update struct {
	Version string
	Update  func() error
}

An Update encapsulates everything needed to perform an update (of envoy configuration). The version string is for logging purposes, the Updator func does the actual work of updating.

type Validatable 

type Validatable interface {
	proto.Message
	Validate() error
}

Not sure if there is a better way to do this, but we cast to this so we can call the generated Validate method.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.