README
¶
Ambex: Ambassador Experimental^H^H^H^H^H^H^H^H^H^H^H^H ADS service
Ambassador prior to v0.50.0 worked by writing out Envoy v1 JSON configuration files, then triggering a hot restart of Envoy. This works, but it has some unpleasant limitations:
- Restarts take awhile, and as a result you can't change the configuration very quickly.
- Restarts can drop connections (cf Envoy #2776; more on this later).
- Envoy itself is deprecating the v1 configuration, and will only support v2 in a bit.
To get around these limitations, and generally go for a better
experience all 'round, we want to switch to the so-called xDS
model,
in which Envoy's configuration is supplied by its various "Discovery
Services": e.g. the CDS
is the Cluster Discovery Service; the
EDS
is the Endpoint Discovery Service. For Ambassador, the
Aggregated Discovery Service or ADS
is the one we want to use --
basically, it brings the other services together under one aegis and
lets you just tell Envoy "get everything dynamically."
However, the whole ADS
thing is a bit of a pain:
- Envoy makes a bidirectional gRPC stream to the
ADS
server. - The
ADS
then makes gRPC calls to the Envoy to feed the Envoy configuration elements, but: - The
ADS
has to carefully order things such that the configuration elements match what Envoy expects for consistency.
Rather than do all that logic by hand, we'll use the Envoy
go-control-plane
[^1] for the heavy lifting. This is also something of a
pain, given that it's not well documented, but here's the deal:
-
The root of the world is a
SnapshotCache
:import github.com/datawire/ambassador/pkg/envoy-control-plane/cache
, then refer tocache.SnapshotCache
.- A collection of internally consistent configuration objects is a
Snapshot
(cache.Snapshot
). Snapshot
s are collected in theSnapshotCache
.- A given
SnapshotCache
can hold configurations for multiple Envoys, identified by the EnvoynodeID
, which must be configured for the Envoy.
-
The
SnapshotCache
can only holdgo-control-plane
configuration objects, so you have to build these up to hand to theSnapshotCache
. -
The gRPC stuff is handled by a
Server
:import github.com/datawire/ambassador/pkg/envoy-control-plane/server
, then refer toserver.Server
.- Our
runManagementServer
function (largely ripped off from thego-control-plane
tests) gets this running. It takes aSnapshotCache
(cleverly calledconfig
for no reason I (Flynn) understand) and a standard GogRPCServer
as arguments. - ALL the gRPC madness is handled by the
Server
, with the assistance of the methods in itscallback
object.
-
Once the
Server
is running, Envoy can open a gRPC stream to it.- On connection, Envoy will get handed the most recent
Snapshot
that theServer
'sSnapshotCache
knows about. - Whenever a newer
Snapshot
is added to theSnapshotCache
, thatSnapshot
will get sent to the Envoy.
- On connection, Envoy will get handed the most recent
-
We manage the
SnapshotCache
by loading Envoy configuration files on disk:- it ignores files that start with a
.
(hidden files) - it interprets
*.json
files as JSON-encoded protobuf - it interprets
*.pb
files as text-encoded protobuf - all other files are ignored As for when it loads those files:
- By default when we get a SIGHUP we reload the configuration.
- When passed the
--watch
argument we reload whenever any file in the directory changes. Be careful about updating files atomically if you use this!
- it ignores files that start with a
[^1]: The Envoy go-control-plane
usually refers to
github.com/envoyproxy/go-control-plane
, but we've "forked" it
as github.com/datawire/ambassador/pkg/envoy-control-plane
in
order to build it against the protobufs for our patched Envoy.
Running Ambex
You'll need the Go toolchain, and will want to have a functioning envoy
.
Then, you can run the ambex CLI using busyambassador
:
go run github.com/datawire/ambassador/cmd/busyambassador ambex ARGs...
If you're on a platform other than GNU/Linux, in order to have a
functioning envoy
, you may want to run all of this in the builder
shell: make shell
.
Try it out
You'll want to run both ambex
and an instance of envoy
with a
boostrap config pointing at that `ambex.
-
First, start the
ambex
:go run github.com/datawire/ambassador/cmd/busyambassador ambex ./example/ambex/
or
go run github.com/datawire/ambassador/cmd/busyambassador ambex --watch ./example/ambex/
-
Second, in another shell, start the
envoy
:envoy -l debug -c ./example/envoy/bootstrap-ads.yaml
You should now be able to run some curls in (yet) another shell:
$ curl localhost:8080/hello
Hello ADS!!!
$ curl localhost:8080/get
{
"args": {},
"headers": {
"Accept": "*/*",
"Connection": "close",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0",
"X-Envoy-Expected-Rq-Timeout-Ms": "15000"
},
"origin": "72.74.69.156",
"url": "http://httpbin.org/get"
}
Edit and/or add more files to the ./example/ambex/
directory in
order to play with more configurations and see them reload
instantaneously (if you used the --watch
flag), or when-triggered
(if you didn't use the --watch
flag; trigger a relead by signaling
the process with killall -HUP ambex
).
Clean up
Kill Ambex and Envoy with Ctrl-C
.
Documentation
¶
Index ¶
- func Clone(src proto.Message) proto.Message
- func Decode(name string) (proto.Message, error)
- func GetAmbassadorDrainTime() time.Duration
- func JoinEdsClusters(ctx context.Context, clusters []ctypes.Resource, ...) (endpoints []ctypes.Resource)
- func JoinEdsClustersV3(ctx context.Context, clusters []ctypes.Resource, ...) (endpoints []ctypes.Resource)
- func ListenerToRdsListener(lnr *api.Listener) (*api.Listener, []*api.RouteConfiguration, error)
- func Main(ctx context.Context, Version string, rawArgs ...string) error
- func Main2(ctx context.Context, Version string, getUsage MemoryGetter, ...) error
- func Merge(to, from proto.Message)
- func Updater(ctx context.Context, updates <-chan Update, getUsage MemoryGetter) error
- func V3ListenerToRdsListener(lnr *v3listener.Listener) (*v3listener.Listener, []*v3route.RouteConfiguration, error)
- type Args
- type Endpoint
- type Endpoints
- type FastpathSnapshot
- type HasherV2
- type HasherV3
- type MemoryGetter
- type Update
- type Validatable
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func GetAmbassadorDrainTime ¶
The GetAmbassadorDrainTime function retuns the AMBASSADOR_DRAIN_TIME env var as a time.Duration
func JoinEdsClusters ¶
func JoinEdsClusters(ctx context.Context, clusters []ctypes.Resource, edsEndpoints map[string]*v2.ClusterLoadAssignment) (endpoints []ctypes.Resource)
JoinEdsClusters will perform an outer join operation between the eds clusters in the supplied clusterlist and the eds endpoint data in the supplied map. It will return a slice of ClusterLoadAssignments (cast to []ctypes.Resource) with endpoint data for all the eds clusters in the supplied list. If there is no map entry for a given cluster, an empty ClusterLoadAssignment will be synthesized. The result is a set of endpoints that are consistent (by the go-control-plane's definition of consistent) with the input clusters.
func JoinEdsClustersV3 ¶
func JoinEdsClustersV3(ctx context.Context, clusters []ctypes.Resource, edsEndpoints map[string]*v3endpoint.ClusterLoadAssignment) (endpoints []ctypes.Resource)
JoinEdsClustersV3 will perform an outer join operation between the eds clusters in the supplied clusterlist and the eds endpoint data in the supplied map. It will return a slice of ClusterLoadAssignments (cast to []ctypes.Resource) with endpoint data for all the eds clusters in the supplied list. If there is no map entry for a given cluster, an empty ClusterLoadAssignment will be synthesized. The result is a set of endpoints that are consistent (by the go-control-plane's definition of consistent) with the input clusters.
func ListenerToRdsListener ¶
ListenerToRdsListener will take a listener definition and extract any inline RouteConfigurations replacing them with a reference to an RDS supplied route configuration. It does not modify the supplied listener, any configuration included in the result is copied from the input.
If the input listener does not match the expected form it is simply copied, i.e. it is the identity transform for any inputs not matching the expected form.
Example Input (that will get transformed in a non-identity way):
a listener configured with an http connection manager
that specifies an http router
that supplies its RouteConfiguration inline via the route_config field
{ "name": "...", ..., "filter_chains": [ { "filter_chain_match": {...}, "filters": [ { "name": "envoy.filters.network.http_connection_manager", "typed_config": { "@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager", "http_filters": [...], "route_config": { "virtual_hosts": [ { "name": "ambassador-listener-8443-*", "domains": ["*"], "routes": [...], } ] } } } ] } ] }
Example Output:
a duplicate listener that defines the "rds" field instead of the "route_config" field
and a list of route configurations
with route_config_name supplied in such a way as to correlate the two together
lnr, routes, err := ListenerToRdsListener(...)
lnr = { "name": "...", ..., "filter_chains": [ { "filter_chain_match": {...}, "filters": [ { "name": "envoy.filters.network.http_connection_manager", "typed_config": { "@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager", "http_filters": [...], "rds": { "config_source": { "ads": {} }, "route_config_name": "ambassador-listener-8443-routeconfig-0" } } } ] } ] }
routes = [ { "name": "ambassador-listener-8443-routeconfig-0", "virtual_hosts": [ { "name": "ambassador-listener-8443-*", "domains": ["*"], "routes": [...], } ] } ]
func Main ¶
NOTE WELL: this Main() does NOT RUN from entrypoint! This one is only relevant if you explicitly run Ambex by hand.
func Main2 ¶
func Main2(ctx context.Context, Version string, getUsage MemoryGetter, fastpathCh <-chan *FastpathSnapshot, rawArgs ...string) error
func Updater ¶
func Updater(ctx context.Context, updates <-chan Update, getUsage MemoryGetter) error
The Updator function will run forever (or until the ctx is canceled) and look for updates on the incoming channel. If memory usage is constrained as reported by the getUsage function, updates will be rate limited to guarantee that there are only so many stale configs in memory at a time. The function assumes updates are cumulative and it will drop old queued updates if a new update arrives.
func V3ListenerToRdsListener ¶
func V3ListenerToRdsListener(lnr *v3listener.Listener) (*v3listener.Listener, []*v3route.RouteConfiguration, error)
V3ListenerToRdsListener is the v3 variety of ListnerToRdsListener
Types ¶
type Endpoint ¶
Endpoint contains the subset of fields we bother to expose.
func (*Endpoint) ToLbEndpoint_v2 ¶
func (e *Endpoint) ToLbEndpoint_v2() *v2endpoint.LbEndpoint
ToLBEndpoint_v2 translates to envoy v2 frinedly form of the Endpoint data.
func (*Endpoint) ToLbEndpoint_v3 ¶
func (e *Endpoint) ToLbEndpoint_v3() *v3endpoint.LbEndpoint
ToLBEndpoint_v3 translates to envoy v3 frinedly form of the Endpoint data.
type Endpoints ¶
The Endpoints struct is how Endpoint data gets communicated to ambex. This is a bit simpler than the envoy endpoint data structures, and also provides us a layer of indirection to buffer us from changes in envoy configuration, e.g. we can switch from v2 to v3 endpoint data, or add v3 endpoint data fairly easily with this layer of indirection.
func (*Endpoints) RoutesString ¶
func (*Endpoints) ToMap_v2 ¶
func (e *Endpoints) ToMap_v2() map[string]*v2.ClusterLoadAssignment
ToMap_v2 produces a map with the envoy v2 friendly forms of all the endpoint data.
func (*Endpoints) ToMap_v3 ¶
func (e *Endpoints) ToMap_v3() map[string]*v3endpointconfig.ClusterLoadAssignment
ToMap_v3 produces a map with the envoy v3 friendly forms of all the endpoint data.
type FastpathSnapshot ¶
FastpathSnapshot holds envoy configuration that bypasses python.
type MemoryGetter ¶
type MemoryGetter func() int
Function type for fetching memory usage as a percentage.
type Update ¶
An Update encapsulates everything needed to perform an update (of envoy configuration). The version string is for logging purposes, the Updator func does the actual work of updating.
type Validatable ¶
Not sure if there is a better way to do this, but we cast to this so we can call the generated Validate method.