experimental

package
v1.17.0-pre.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 2, 2024 License: Apache-2.0 Imports: 54 Imported by: 0

README

Experimental load-balancing control-plane

This package implements a replacement for ServiceManager and ServiceCache. It aims to simplify management of the load-balancing state by implementing it as the StateDB tables services, frontends and backends. The reconciliation of the state towards BPF maps is implemented asynchronously with the StateDB reconciler.

Modifying the state is done via the Writer API. It ensures consistency and manages the references between services, frontends and backends.

The basic architecture looks as follows:

  [Data source A]       [Data source B]      [Health checker]    
        \                     |                  /   ^     
         ----------------.    v   .--------------   /
                          [Writer]                 /
                         /    |   \               /
              .---------'     v    '------.      /
         [Services]      [Frontends]    [Backends]
          /                  | ^              |
         /                   v |              v
    [ Observer(s) ]      [Reconciler]    [ Envoy sync ]
                              |
                              v
                          [BPF maps]

The different data sources insert data using Writer.UpsertService, Writer.UpsertFrontend, etc. methods.

A health checker observes the backends table to find targets to health check and uses Writer.SetBackendHealth to update the state.

The BPF reconciler watches the frontend table (service and backend objects are referenced by the frontend object) and reconcile updates towards the BPF maps. The reconciliation status is written back to frontends.

There can be any number of observers to these tables. For example components that need access to Services can watch the Table[Service] instead of e.g. going via Resource[corev1.Service]. For L7 proxying we can sync the backends towards Envoy asynchronously by watching changes to the backends table.

This architecture enables:

  • Easy addition of new data sources
  • Ability to observe changes to the data at coarse or fine granularity via StateDB watch channels
  • Separation of concerns: the error handling is performed by the reconciler and low-level errors are not bubbled up to data sources (which wouldn't know how to handle it). Readers are not in the critical path.

Running

The experimental control-plane can be enabled either via the helm option loadBalancer.experimental or via the command-line flag enable-experimental-lb. The latter can be set with cilium-cli: cilium config set enable-experimental-lb true.

Enabling the experimental control-plane will swap the LBMap instance given to ServiceManager with a fake one. This retains all the agent side functionality except for the updates towards BPF maps which are instead done by the experimental control-plane.

Once running the state can be inspected with:

  $ cilium-dbg shell 
  > db show frontends
  > db show backends
  > db show services
  > db prefix health agent.controlplane.loadbalancer-experimental
  > lb-maps dump

Integration testing

The new architecture makes it easier to write integration tests due to the decoupling. An integration test for a data source can depend just on the tables & writer and verify that the tables are correctly populated without having to mock other features or the BPF operations. An example of this sort of test is in writer_test.go.

A more "end-to-end" style test can be found in k8s_test.go that tests going from Kubernetes objects specified in YAML to the BPF map contents.

To run the privileged tests:

  $ go test -c
  $ PRIVILEGED_TESTS=1 sudo -E ./experimental.test -test.run . -test.v -test.count 1

Unimplemented features

This section lists the major unimplemented features and some thoughts on how we'd go about implementing them.

Local Redirect Policies

These are currently implemented by pkg/redirectpolicy. The manager there subscribes to pods and redirect policies and create a -local service with the backends derived from the matched local pods.

With the new architecture this should be implemented as a controller that watches Table[Pod] and Table[CiliumLocalRedirectPolicy] and updates the load-balancing state accordingly when they change. The controller should be as stateless as possible.

Still TBD is the design for how services would be overridden to redirect traffic to a local pod. The essential problem is that there's two sets of backends for the service: the normal backends and the LRP backends, and it should be possible to return to normal when the LRP is removed. One option is to use a generated service name for the set of LRP backends and then set this in the Service object. The matching of frontends with backends would then see if this field is set and use the alternative service name. The challenge here is how to make it easy for observers of the tables, e.g. how the L7 proxy would be able to correctly look up backends for a specific service.

L7 proxy

A CiliumEnvoyConfig CRD specifies that traffic for a specific service should be redirected to an Envoy instance. This is accomplished by setting the L7 proxy port in the services BPF map value. The current implementation lives in pkg/ciliumenvoyconfig and on Resource[CiliumEnvoyConfig] event calls to ServiceManager to add the L7 proxy port to matching services.

As with LRPs it should be possible to reimplement this in mostly stateless way by switching Resource[CiliumEnvoyConfig] to Table[CiliumEnvoyConfig] and having the controller watch Table[Service] for matches and updating when the service matches with a CiliumEnvoyConfig. To avoid the intermediate state where a service is missing the L7 proxy port we can implement a "service hook" that is invoked when a service is upserted to do a query against Table[CiliumEnvoyConfig] and fill it in on the fly.

REST API (/service)

The REST API handlers are currently implemented against ServiceManager. Implementing the inspection of the state is relatively easy thanks to the generic StateDB HTTP API. Handlers for modifying the state (e.g. for lb-only use) are trickier due to the semantic differences. In the new design the "primary key" is the L3n4Addr rather than the ID, so changes would be needed to the REST API to accomodate this. There's relatively few users of this and the message has been that the change from IDs to addresses would simplify their lives.

Additional question here is the change to the restoration of data from BPF maps. The new implementation only restores the IDs (in order to reuse them and avoid connection disruption), but not all the rest of the data. This means that in lb-only use the services and backends need to be restored when the agent has restarted. Likely it would make sense to rethink the API to allow implementation of "data sources" via the REST API that can be resynced, e.g. move into a more of a pull-style API.

ClusterMesh

ClusterMesh is implemented by merging the external services and endpoints in ServiceCache (Merge* methods). This can be now implemented more directly with the Writer API. The ClusterMesh services and backends are essentially the same as those coming from Kubernetes and do not require any special handling.

One notable requirement for ClusterMesh is the need to prune non-global services before ClusterMesh-sourced services are initialized. See cf4279c68202bae83917b65b8e7da21e20869def for context. Yet unclear how to cleanly implement this. The reconciler currently won't perform the Prune() operation if there are any pending initializers.

ServiceCache replacement

ServiceCache in addition to merging Services with Endpoints and forwarding as events to a handler, it has a set of getters and a Notifications() stream (used by policy engine).

These are all easy to replace by queries against the service/frontend/backend tables.

kvstore data source
Misc features
  • ServiceAffinity/"Preferred"
  • TopologyAware/Zones
  • LoopbackHostPort

Documentation

Overview

Package experimental contains a new experimental load-balancer control-plane based on StateDB. It aims to simplify the control-plane down to fewer layers and open up the ability to modify and observe frontends and backends for variaty of use-cases without the need to special-case them.

Index

Constants

View Source 
const (
	BackendTableName = "backends"
)
View Source 
const (
	FrontendTableName = "frontends"
)
View Source 
const (
	ServiceTableName = "services"
)

Variables

View Source 
var (
	BackendByAddress = backendAddrIndex.Query

	BackendByServiceName = backendServiceIndex.Query
)
View Source 
var (
	FrontendByAddress = frontendAddressIndex.Query

	FrontendByServiceName = frontendServiceIndex.Query
)
View Source 
var Cell = cell.Module(
	"loadbalancer-experimental",
	"Experimental load-balancing control-plane",

	cell.Config(DefaultConfig),
	cell.ProvidePrivate(newExternalConfig),

	TablesCell,

	ReflectorCell,

	cell.ProvidePrivate(resourcesToStreams),

	ReconcilerCell,

	cell.ProvidePrivate(newLBMaps, newLBMapsConfig),
	cell.Provide(newLBMapsCommand),
)
View Source 
var DefaultConfig = Config{
	EnableExperimentalLB: false,
	RetryBackoffMin:      50 * time.Millisecond,
	RetryBackoffMax:      time.Minute,
}
View Source 
var (
	// ErrServiceNotFound occurs when a frontend is being upserted that refers to
	// a non-existing service.
	ErrServiceNotFound = errors.New("service not found")
)
View Source 
var ReconcilerCell = cell.Module(
	"reconciler",
	"Reconciles load-balancing state with BPF maps",

	cell.Provide(
		newBPFOps,
		newBPFReconciler,
	),
	cell.Invoke(

		func(reconciler.Reconciler[*Frontend]) {},
	),
)

ReconcilerCell reconciles the load-balancing state with the BPF maps.

View Source 
var ReflectorCell = cell.Module(
	"reflector",
	"Reflects load-balancing state from Kubernetes",

	cell.Invoke(registerK8sReflector),
)

ReflectorCell reflects Kubernetes Service and EndpointSlice objects to the load-balancing tables.

Note that this implementation uses Resource[*Service] and Resource[*Endpoints], which is not the desired end-game as we'll hold onto the same data multiple times. We should instead have a reflector that is built directly on the client-go reflector (k8s.RegisterReflector) and not populate an intermediate cache.Store. But as we're still experimenting it's easier to build on what already exists.

View Source 
var (
	ServiceByName = serviceNameIndex.Query
)
View Source 
var TablesCell = cell.Module(
	"tables",
	"Experimental load-balancing control-plane",

	cell.ProvidePrivate(
		NewServicesTable,
		NewFrontendsTable,
		NewBackendsTable,
	),

	cell.Provide(

		NewWriter,

		statedb.RWTable[*Service].ToTable,
		statedb.RWTable[*Frontend].ToTable,
		statedb.RWTable[*Backend].ToTable,
	),
)

TablesCell provides the Writer API for configuring load-balancing and the Table[*Service], Table[*Frontend] and Table[*Backend] for read-only access to load-balancing state.

Functions

func FastCheckEmptyTablesAndState 

func FastCheckEmptyTablesAndState(db *statedb.DB, writer *Writer, bo *BPFOps) bool

func FastCheckTables 

func FastCheckTables(db *statedb.DB, writer *Writer, expectedFrontends int, lastPendingRevision statedb.Revision) (reconciled bool, nextRevision statedb.Revision)

func NewBackendsTable 

func NewBackendsTable(db *statedb.DB) (statedb.RWTable[*Backend], error)

func NewFrontendsTable 

func NewFrontendsTable(db *statedb.DB) (statedb.RWTable[*Frontend], error)

func NewServicesTable 

func NewServicesTable(db *statedb.DB) (statedb.RWTable[*Service], error)

Types

type BPFLBMaps 

type BPFLBMaps struct {
	// Pinned if true will pin the maps to a file. Tests may turn this off.
	Pinned bool

	Cfg LBMapsConfig
	// contains filtered or unexported fields
}

func (*BPFLBMaps) DeleteAffinityMatch 

func (r *BPFLBMaps) DeleteAffinityMatch(key *lbmap.AffinityMatchKey) error

DeleteAffinityMatch implements lbmaps.

func (*BPFLBMaps) DeleteBackend 

func (r *BPFLBMaps) DeleteBackend(key lbmap.BackendKey) error

DeleteBackend implements lbmaps.

func (*BPFLBMaps) DeleteMaglev 

func (r *BPFLBMaps) DeleteMaglev(key lbmap.MaglevOuterKey, ipv6 bool) error

DeleteMaglev implements lbmaps.

func (*BPFLBMaps) DeleteRevNat 

func (r *BPFLBMaps) DeleteRevNat(key lbmap.RevNatKey) error

DeleteRevNat implements lbmaps.

func (*BPFLBMaps) DeleteService 

func (r *BPFLBMaps) DeleteService(key lbmap.ServiceKey) error

DeleteService implements lbmaps.

func (*BPFLBMaps) DeleteSourceRange 

func (r *BPFLBMaps) DeleteSourceRange(key lbmap.SourceRangeKey) error

DeleteSourceRange implements lbmaps.

func (*BPFLBMaps) DumpAffinityMatch 

func (r *BPFLBMaps) DumpAffinityMatch(cb func(*lbmap.AffinityMatchKey, *lbmap.AffinityMatchValue)) error

DumpAffinityMatch implements lbmaps.

func (*BPFLBMaps) DumpBackend 

func (r *BPFLBMaps) DumpBackend(cb func(lbmap.BackendKey, lbmap.BackendValue)) error

DumpBackend implements lbmaps.

func (*BPFLBMaps) DumpMaglev 

func (r *BPFLBMaps) DumpMaglev(cb func(lbmap.MaglevOuterKey, lbmap.MaglevOuterVal, lbmap.MaglevInnerKey, *lbmap.MaglevInnerVal, bool)) error

func (*BPFLBMaps) DumpRevNat 

func (r *BPFLBMaps) DumpRevNat(cb func(lbmap.RevNatKey, lbmap.RevNatValue)) error

DumpRevNat implements lbmaps.

func (*BPFLBMaps) DumpService 

func (r *BPFLBMaps) DumpService(cb func(lbmap.ServiceKey, lbmap.ServiceValue)) error

DumpService implements lbmaps.

func (*BPFLBMaps) DumpSourceRange 

func (r *BPFLBMaps) DumpSourceRange(cb func(lbmap.SourceRangeKey, *lbmap.SourceRangeValue)) error

DumpSourceRange implements lbmaps.

func (*BPFLBMaps) IsEmpty 

func (r *BPFLBMaps) IsEmpty() bool

IsEmpty implements lbmaps.

func (*BPFLBMaps) Start 

func (r *BPFLBMaps) Start(cell.HookContext) error

Start implements cell.HookInterface.

func (*BPFLBMaps) Stop 

func (r *BPFLBMaps) Stop(cell.HookContext) error

Stop implements cell.HookInterface.

func (*BPFLBMaps) UpdateAffinityMatch 

func (r *BPFLBMaps) UpdateAffinityMatch(key *lbmap.AffinityMatchKey, value *lbmap.AffinityMatchValue) error

UpdateAffinityMatch implements lbmaps.

func (*BPFLBMaps) UpdateBackend 

func (r *BPFLBMaps) UpdateBackend(key lbmap.BackendKey, value lbmap.BackendValue) error

UpdateBackend implements lbmaps.

func (*BPFLBMaps) UpdateMaglev 

func (r *BPFLBMaps) UpdateMaglev(key lbmap.MaglevOuterKey, backendIDs []loadbalancer.BackendID, ipv6 bool) error

UpdateMaglev implements lbmaps.

func (*BPFLBMaps) UpdateRevNat 

func (r *BPFLBMaps) UpdateRevNat(key lbmap.RevNatKey, value lbmap.RevNatValue) error

UpdateRevNat4 implements lbmaps.

func (*BPFLBMaps) UpdateService 

func (r *BPFLBMaps) UpdateService(key lbmap.ServiceKey, value lbmap.ServiceValue) error

UpdateService implements lbmaps.

func (*BPFLBMaps) UpdateSourceRange 

func (r *BPFLBMaps) UpdateSourceRange(key lbmap.SourceRangeKey, value *lbmap.SourceRangeValue) error

UpdateSourceRange implements lbmaps.

type BPFOps 

type BPFOps struct {
	LBMaps LBMaps
	// contains filtered or unexported fields
}

func (*BPFOps) Delete 

func (ops *BPFOps) Delete(_ context.Context, _ statedb.ReadTxn, fe *Frontend) error

Delete implements reconciler.Operations.

func (*BPFOps) Prune 

func (ops *BPFOps) Prune(_ context.Context, _ statedb.ReadTxn, _ iter.Seq2[*Frontend, statedb.Revision]) error

Prune implements reconciler.Operations.

func (*BPFOps) Update 

func (ops *BPFOps) Update(_ context.Context, _ statedb.ReadTxn, fe *Frontend) error

Update implements reconciler.Operations.

type Backend 

type Backend struct {
	loadbalancer.L3n4Addr

	// State is the learned state of the backend that combines the state of the
	// instances and the results of health checking.
	State loadbalancer.BackendState

	// Node hosting this backend. This is used to determine backends local to
	// a node.
	NodeName string

	// Zone where backend is located.
	ZoneID uint8

	// Instances of this backend. A backend is always linked to a specific
	// service and the instances may call the backend by different name
	// (PortName) or they may come from  differents sources.
	Instances part.Map[loadbalancer.ServiceName, BackendInstance]

	// Properties are additional untyped properties that can carry feature
	// specific metadata about the backend.
	Properties part.Map[string, any]
}

Backend is a composite of the per-service backend instances that share the same IP address and port.

func (*Backend) Clone 

func (be *Backend) Clone() *Backend

Clone returns a shallow clone of the backend.

func (*Backend) String 

func (be *Backend) String() string

func (*Backend) TableHeader 

func (be *Backend) TableHeader() []string

func (*Backend) TableRow 

func (be *Backend) TableRow() []string

type BackendInstance 

type BackendInstance struct {
	// PortName is the frontend port name used for filtering the backends
	// associated with a service.
	PortName string

	// Weight is the load-balancing weight for this backend in association
	// with a specific service.
	Weight uint16

	// Source is the data source from which this backend came from.
	Source source.Source

	// State is the backend's state as defined by the data source. This is
	// taken as input along with learned state (e.g. via health checking) to
	// construct the definite state.
	State loadbalancer.BackendState
}

BackendInstance defines the backend's properties associated with a specific service.

type BackendParams 

type BackendParams struct {
	loadbalancer.L3n4Addr

	// PortName is the frontend port name. If a frontend has specified a port name
	// only the backends with matching port name are selected.
	PortName string

	// Weight of backend for load-balancing.
	Weight uint16

	// Node hosting this backend. This is used to determine backends local to
	// a node.
	NodeName string

	// Zone where backend is located.
	ZoneID uint8

	// State of the backend for load-balancing service traffic
	State loadbalancer.BackendState
}

BackendParams defines the parameters of a backend for insertion into the backends table.

type BackendWithRevision 

type BackendWithRevision struct {
	*Backend
	Revision statedb.Revision
}

type Config 

type Config struct {
	EnableExperimentalLB bool          `mapstructure:"enable-experimental-lb"`
	RetryBackoffMin      time.Duration `mapstructure:"lb-retry-backoff-min"`
	RetryBackoffMax      time.Duration `mapstructure:"lb-retry-backoff-max"`
}

func (Config) Flags 

func (def Config) Flags(flags *pflag.FlagSet)

type ExternalConfig 

type ExternalConfig struct {
	ExternalClusterIP        bool
	EnableSessionAffinity    bool
	NodePortMin, NodePortMax uint16
	NodePortAlg              string
	MaglevTableSize          int
}

ExternalConfig are configuration options derived from external sources such as DaemonConfig. This avoids direct access of larger configuration structs.

type FakeLBMaps 

type FakeLBMaps struct {
	// contains filtered or unexported fields
}

func (*FakeLBMaps) DeleteAffinityMatch 

func (f *FakeLBMaps) DeleteAffinityMatch(key *lbmap.AffinityMatchKey) error

DeleteAffinityMatch implements lbmaps.

func (*FakeLBMaps) DeleteBackend 

func (f *FakeLBMaps) DeleteBackend(key lbmap.BackendKey) error

DeleteBackend implements lbmaps.

func (*FakeLBMaps) DeleteMaglev 

func (f *FakeLBMaps) DeleteMaglev(key lbmap.MaglevOuterKey, ipv6 bool) error

DeleteMaglev implements lbmaps.

func (*FakeLBMaps) DeleteRevNat 

func (f *FakeLBMaps) DeleteRevNat(key lbmap.RevNatKey) error

DeleteRevNat implements lbmaps.

func (*FakeLBMaps) DeleteService 

func (f *FakeLBMaps) DeleteService(key lbmap.ServiceKey) error

DeleteService implements lbmaps.

func (*FakeLBMaps) DeleteSourceRange 

func (f *FakeLBMaps) DeleteSourceRange(key lbmap.SourceRangeKey) error

DeleteSourceRange implements lbmaps.

func (*FakeLBMaps) DumpAffinityMatch 

func (f *FakeLBMaps) DumpAffinityMatch(cb func(*lbmap.AffinityMatchKey, *lbmap.AffinityMatchValue)) error

DumpAffinityMatch implements lbmaps.

func (*FakeLBMaps) DumpBackend 

func (f *FakeLBMaps) DumpBackend(cb func(lbmap.BackendKey, lbmap.BackendValue)) error

DumpBackend implements lbmaps.

func (*FakeLBMaps) DumpMaglev 

func (f *FakeLBMaps) DumpMaglev(cb func(lbmap.MaglevOuterKey, lbmap.MaglevOuterVal, lbmap.MaglevInnerKey, *lbmap.MaglevInnerVal, bool)) error

func (*FakeLBMaps) DumpRevNat 

func (f *FakeLBMaps) DumpRevNat(cb func(lbmap.RevNatKey, lbmap.RevNatValue)) error

DumpRevNat implements lbmaps.

func (*FakeLBMaps) DumpService 

func (f *FakeLBMaps) DumpService(cb func(lbmap.ServiceKey, lbmap.ServiceValue)) error

DumpService implements lbmaps.

func (*FakeLBMaps) DumpSourceRange 

func (f *FakeLBMaps) DumpSourceRange(cb func(lbmap.SourceRangeKey, *lbmap.SourceRangeValue)) error

DumpSourceRange implements lbmaps.

func (*FakeLBMaps) IsEmpty 

func (f *FakeLBMaps) IsEmpty() bool

IsEmpty implements lbmaps.

func (*FakeLBMaps) UpdateAffinityMatch 

func (f *FakeLBMaps) UpdateAffinityMatch(key *lbmap.AffinityMatchKey, value *lbmap.AffinityMatchValue) error

UpdateAffinityMatch implements lbmaps.

func (*FakeLBMaps) UpdateBackend 

func (f *FakeLBMaps) UpdateBackend(key lbmap.BackendKey, value lbmap.BackendValue) error

UpdateBackend implements lbmaps.

func (*FakeLBMaps) UpdateMaglev 

func (f *FakeLBMaps) UpdateMaglev(key lbmap.MaglevOuterKey, backendIDs []loadbalancer.BackendID, ipv6 bool) error

UpdateMaglev implements lbmaps.

func (*FakeLBMaps) UpdateRevNat 

func (f *FakeLBMaps) UpdateRevNat(key lbmap.RevNatKey, value lbmap.RevNatValue) error

UpdateRevNat implements lbmaps.

func (*FakeLBMaps) UpdateService 

func (f *FakeLBMaps) UpdateService(key lbmap.ServiceKey, value lbmap.ServiceValue) error

UpdateService implements lbmaps.

func (*FakeLBMaps) UpdateSourceRange 

func (f *FakeLBMaps) UpdateSourceRange(key lbmap.SourceRangeKey, value *lbmap.SourceRangeValue) error

UpdateSourceRange implements lbmaps.

type FaultyLBMaps 

type FaultyLBMaps struct {
	// contains filtered or unexported fields
}

func (*FaultyLBMaps) DeleteAffinityMatch 

func (f *FaultyLBMaps) DeleteAffinityMatch(key *lbmap.AffinityMatchKey) error

DeleteAffinityMatch implements lbmaps.

func (*FaultyLBMaps) DeleteBackend 

func (f *FaultyLBMaps) DeleteBackend(key lbmap.BackendKey) error

DeleteBackend implements lbmaps.

func (*FaultyLBMaps) DeleteMaglev 

func (f *FaultyLBMaps) DeleteMaglev(key lbmap.MaglevOuterKey, ipv6 bool) error

DeleteMaglev implements lbmaps.

func (*FaultyLBMaps) DeleteRevNat 

func (f *FaultyLBMaps) DeleteRevNat(key lbmap.RevNatKey) error

DeleteRevNat implements lbmaps.

func (*FaultyLBMaps) DeleteService 

func (f *FaultyLBMaps) DeleteService(key lbmap.ServiceKey) error

DeleteService implements lbmaps.

func (*FaultyLBMaps) DeleteSourceRange 

func (f *FaultyLBMaps) DeleteSourceRange(key lbmap.SourceRangeKey) error

DeleteSourceRange implements lbmaps.

func (*FaultyLBMaps) DumpAffinityMatch 

func (f *FaultyLBMaps) DumpAffinityMatch(cb func(*lbmap.AffinityMatchKey, *lbmap.AffinityMatchValue)) error

DumpAffinityMatch implements lbmaps.

func (*FaultyLBMaps) DumpBackend 

func (f *FaultyLBMaps) DumpBackend(cb func(lbmap.BackendKey, lbmap.BackendValue)) error

DumpBackend implements lbmaps.

func (*FaultyLBMaps) DumpMaglev 

func (f *FaultyLBMaps) DumpMaglev(cb func(lbmap.MaglevOuterKey, lbmap.MaglevOuterVal, lbmap.MaglevInnerKey, *lbmap.MaglevInnerVal, bool)) error

DumpMaglev implements lbmaps.

func (*FaultyLBMaps) DumpRevNat 

func (f *FaultyLBMaps) DumpRevNat(cb func(lbmap.RevNatKey, lbmap.RevNatValue)) error

DumpRevNat implements lbmaps.

func (*FaultyLBMaps) DumpService 

func (f *FaultyLBMaps) DumpService(cb func(lbmap.ServiceKey, lbmap.ServiceValue)) error

DumpService implements lbmaps.

func (*FaultyLBMaps) DumpSourceRange 

func (f *FaultyLBMaps) DumpSourceRange(cb func(lbmap.SourceRangeKey, *lbmap.SourceRangeValue)) error

DumpSourceRange implements lbmaps.

func (*FaultyLBMaps) IsEmpty 

func (f *FaultyLBMaps) IsEmpty() bool

IsEmpty implements lbmaps.

func (*FaultyLBMaps) UpdateAffinityMatch 

func (f *FaultyLBMaps) UpdateAffinityMatch(key *lbmap.AffinityMatchKey, value *lbmap.AffinityMatchValue) error

UpdateAffinityMatch implements lbmaps.

func (*FaultyLBMaps) UpdateBackend 

func (f *FaultyLBMaps) UpdateBackend(key lbmap.BackendKey, value lbmap.BackendValue) error

UpdateBackend implements lbmaps.

func (*FaultyLBMaps) UpdateMaglev 

func (f *FaultyLBMaps) UpdateMaglev(key lbmap.MaglevOuterKey, backendIDs []loadbalancer.BackendID, ipv6 bool) error

UpdateMaglev implements lbmaps.

func (*FaultyLBMaps) UpdateRevNat 

func (f *FaultyLBMaps) UpdateRevNat(key lbmap.RevNatKey, value lbmap.RevNatValue) error

UpdateRevNat implements lbmaps.

func (*FaultyLBMaps) UpdateService 

func (f *FaultyLBMaps) UpdateService(key lbmap.ServiceKey, value lbmap.ServiceValue) error

UpdateService implements lbmaps.

func (*FaultyLBMaps) UpdateSourceRange 

func (f *FaultyLBMaps) UpdateSourceRange(key lbmap.SourceRangeKey, value *lbmap.SourceRangeValue) error

UpdateSourceRange implements lbmaps.

type Frontend 

type Frontend struct {
	FrontendParams

	// Status is the reconciliation status for this frontend and
	// reflects whether or not the frontend and the associated backends
	// have been reconciled with the BPF maps.
	// Managed by [Writer].
	Status reconciler.Status

	// Backends associated with the frontend.
	Backends []BackendWithRevision
	// contains filtered or unexported fields
}

func (*Frontend) Clone 

func (fe *Frontend) Clone() *Frontend

func (*Frontend) Service 

func (fe *Frontend) Service() *Service

func (*Frontend) TableHeader 

func (fe *Frontend) TableHeader() []string

func (*Frontend) TableRow 

func (fe *Frontend) TableRow() []string

type FrontendParams 

type FrontendParams struct {
	// Frontend address and port
	Address loadbalancer.L3n4Addr

	// Service type (e.g. ClusterIP, NodePort, ...)
	Type loadbalancer.SVCType

	// Name of the associated service
	ServiceName loadbalancer.ServiceName

	// PortName if set will select only backends with matching
	// port name.
	PortName loadbalancer.FEPortName

	// ServicePort is the associated "ClusterIP" port of this frontend.
	// Same as [Address.L4Addr.Port] except when [Type] NodePort or
	// LoadBalancer. This is used to match frontends with the [Ports] of
	// [Service.ProxyRedirect].
	ServicePort uint16
}

FrontendParams defines the static parameters of a frontend. This is separate from Frontend to clearly separate which fields can be manipulated and which are internally managed by Writer.

type LBMaps 

type LBMaps interface {

	// TODO rest of the maps:
	// Maglev, SockRevNat, SkipLB
	IsEmpty() bool
	// contains filtered or unexported methods
}

LBMaps defines the map operations performed by the reconciliation. Depending on this interface instead of on the underlying maps allows testing the implementation with a fake map or injected errors.

func NewFakeLBMaps 

func NewFakeLBMaps() LBMaps

type LBMapsConfig 

type LBMapsConfig struct {
	MaxSockRevNatMapEntries                                         int
	ServiceMapMaxEntries, BackendMapMaxEntries, RevNatMapMaxEntries int
	AffinityMapMaxEntries                                           int
	SourceRangeMapMaxEntries                                        int
	MaglevMapMaxEntries                                             int
	MaglevTableSize                                                 int
}

LBMapsConfig specifies the configuration for the load-balancing BPF maps.

type MapDump 

type MapDump = string

MapDump is a dump of a BPF map. These are generated by the dump() method, which solely defines the format.

func DumpLBMaps 

func DumpLBMaps(lbmaps LBMaps, feAddr loadbalancer.L3n4Addr, sanitizeIDs bool, customIPString func(net.IP) string) (out []MapDump)

DumpLBMaps the load-balancing maps into a concise format for assertions in tests.

type ProxyRedirect 

type ProxyRedirect struct {
	ProxyPort uint16

	// Ports if non-empty will only redirect a frontend with a matching port.
	Ports []uint16
}

func (*ProxyRedirect) Equal 

func (pr *ProxyRedirect) Equal(other *ProxyRedirect) bool

func (*ProxyRedirect) Redirects 

func (pr *ProxyRedirect) Redirects(port uint16) bool

func (*ProxyRedirect) String 

func (pr *ProxyRedirect) String() string

type Service 

type Service struct {
	// Name is the fully qualified service name, e.g. (<cluster>/)<namespace>/<name>.
	Name loadbalancer.ServiceName

	// Source is the data source from which this service was ingested from.
	Source source.Source

	// Labels associated with the service.
	Labels labels.Labels

	// Annotations associated with this service.
	Annotations map[string]string

	// NatPolicy defines whether we need NAT46/64 translation for backends.
	NatPolicy loadbalancer.SVCNatPolicy

	// ExtTrafficPolicy controls how backends are selected for North-South traffic.
	// If set to "Local", only node-local backends are chosen.
	ExtTrafficPolicy loadbalancer.SVCTrafficPolicy

	// IntTrafficPolicy controls how backends are selected for East-West traffic.
	// If set to "Local", only node-local backends are chosen.
	IntTrafficPolicy loadbalancer.SVCTrafficPolicy

	SessionAffinity        bool
	SessionAffinityTimeout time.Duration

	// ProxyRedirect if non-nil redirects the traffic going to the frontends
	// towards a locally running proxy.
	ProxyRedirect *ProxyRedirect

	// HealthCheckNodePort defines on which port the node runs a HTTP health
	// check server which may be used by external loadbalancers to determine
	// if a node has local backends. This will only have effect if both
	// LoadBalancerIPs is not empty and ExtTrafficPolicy is SVCTrafficPolicyLocal.
	HealthCheckNodePort uint16

	// LoopbackHostPort defines that HostPort frontends for this service should
	// only be exposed internally to the node.
	LoopbackHostPort bool

	// SourceRanges if non-empty will restrict access to the service to the specified
	// client addresses.
	SourceRanges []cidr.CIDR

	// Properties are additional untyped properties that can carry feature
	// specific metadata about the service.
	Properties part.Map[string, any]
}

Service defines the common properties for a load-balancing service. Associated with a service are a set of frontends that receive the traffic, and a set of backends to which the traffic is directed. A single frontend can map to a partial subset of backends depending on its properties.

func (*Service) Clone 

func (svc *Service) Clone() *Service

Clone returns a shallow clone of the service, e.g. for updating a service with UpsertService. Fields that are references (e.g. Labels or Annotations) must be further cloned if mutated.

func (*Service) TableHeader 

func (svc *Service) TableHeader() []string

func (*Service) TableRow 

func (svc *Service) TableRow() []string

type ServiceHook 

type ServiceHook = func(txn statedb.ReadTxn, svc *Service)

ServiceHook is a function invoked when a frontend has been updated. A hook can manipulate the frontend before the changes are seen by other components.

The main use-case for a hook is to set e.g. L7 proxy port before the frontend is reconciled and thus avoid unnecessary work.

For consistency the hook should only access StateDB tables via the provided read transaction.

type StreamsOut 

type StreamsOut struct {
	cell.Out
	ServicesStream  stream.Observable[resource.Event[*slim_corev1.Service]]
	EndpointsStream stream.Observable[resource.Event[*k8s.Endpoints]]
	PodsStream      stream.Observable[resource.Event[*slim_corev1.Pod]]
}

type TestConfig 

type TestConfig struct {
	TestFaultProbability float32 `mapstructure:"lb-test-fault-probability"`

	// NodePortAlg mirrors option.Config.NodePortAlg. This can be removed when the NodePort config
	// flags move away from option.DaemonConfig and can thus be set directly.
	NodePortAlg string `mapstructure:"node-port-algorithm"`
}

TestConfig are the configuration options for testing. Only provided by tests and not present in the agent.

func (TestConfig) Flags 

func (def TestConfig) Flags(flags *pflag.FlagSet)

type WriteTxn 

type WriteTxn struct {
	statedb.WriteTxn
}

type Writer 

type Writer struct {
	// contains filtered or unexported fields
}

Writer provides validated write access to the service load-balancing state.

func NewWriter 

func NewWriter(p writerParams) (*Writer, error)

func (*Writer) Backends 

func (w *Writer) Backends() statedb.Table[*Backend]

Backends returns the backend table for reading. Convenience method for reducing dependencies.

func (*Writer) DebugDump 

func (w *Writer) DebugDump(txn statedb.ReadTxn, to io.Writer)

func (*Writer) DeleteBackendsBySource 

func (w *Writer) DeleteBackendsBySource(txn WriteTxn, source source.Source) error

func (*Writer) DeleteServiceAndFrontends 

func (w *Writer) DeleteServiceAndFrontends(txn WriteTxn, name loadbalancer.ServiceName) error

func (*Writer) DeleteServicesBySource 

func (w *Writer) DeleteServicesBySource(txn WriteTxn, source source.Source) error

DeleteServicesBySource deletes all services from the specific source. This is used to implement "resynchronization", for example with K8s when the Watch() call fails and we need to start over with a List().

func (*Writer) Frontends 

func (w *Writer) Frontends() statedb.Table[*Frontend]

Frontends returns the frontend table for reading. Convenience method for reducing dependencies.

func (*Writer) IsEnabled 

func (w *Writer) IsEnabled() bool

func (*Writer) ReadTxn 

func (w *Writer) ReadTxn() statedb.ReadTxn

ReadTxn returns a StateDB read transaction. Convenience method to be used with the above table getters.

func (*Writer) RegisterInitializer 

func (w *Writer) RegisterInitializer(name string) (complete func(WriteTxn))

RegisterInitializer registers a component as an initializer to the load-balancing tables. This blocks pruning of data until this and all other registered initializers have called the returned 'complete' function.

func (*Writer) ReleaseBackend 

func (w *Writer) ReleaseBackend(txn WriteTxn, name loadbalancer.ServiceName, addr loadbalancer.L3n4Addr) error

func (*Writer) ReleaseBackendsFromSource 

func (w *Writer) ReleaseBackendsFromSource(txn WriteTxn, name loadbalancer.ServiceName, source source.Source) error

func (*Writer) Services 

func (w *Writer) Services() statedb.Table[*Service]

Services returns the service table for reading. Convenience method for reducing dependencies.

func (*Writer) SetBackendHealth 

func (w *Writer) SetBackendHealth(txn WriteTxn, addr loadbalancer.L3n4Addr, healthy bool) error

func (*Writer) SetBackends 

func (w *Writer) SetBackends(txn WriteTxn, name loadbalancer.ServiceName, source source.Source, bes ...BackendParams) error

SetBackends sets the backends associated with a service. Existing backends from this source that are associated with the service but are not given are released.

func (*Writer) UpsertBackends 

func (w *Writer) UpsertBackends(txn WriteTxn, serviceName loadbalancer.ServiceName, source source.Source, bes ...BackendParams) error

UpsertBackends adds/updates backends for the given service.

func (*Writer) UpsertFrontend 

func (w *Writer) UpsertFrontend(txn WriteTxn, params FrontendParams) (old *Frontend, err error)

func (*Writer) UpsertService 

func (w *Writer) UpsertService(txn WriteTxn, svc *Service) (old *Service, err error)

func (*Writer) UpsertServiceAndFrontends 

func (w *Writer) UpsertServiceAndFrontends(txn WriteTxn, svc *Service, fes ...FrontendParams) error

UpsertServiceAndFrontends upserts the service and updates the set of associated frontends. Any frontends that do not exist in the new set are deleted.

func (*Writer) WriteTxn 

func (w *Writer) WriteTxn(extraTables ...statedb.TableMeta) WriteTxn

WriteTxn returns a write transaction against services & backends and other additional tables to be used with the methods of Writer. The returned transaction MUST be Abort()'ed or Commit()'ed.

Source Files

View all Source files

Directories

Path Synopsis
cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.