Documentation ¶
Index ¶
- Constants
- func HasAlwaysMigrateLabel(obj metav1.ObjectMetaAccessor) bool
- func HasAutoscalingEnabled(obj metav1.ObjectMetaAccessor) bool
- type AgentDesc
- type AgentIdentification
- type AgentIdentificationMessage
- type AgentMessage
- type AgentRequest
- type AgentResourceMessage
- type DownscaleResult
- type InformantDesc
- type InformantHealthCheckResp
- type InformantMetricsMethod
- type InformantProtoVersion
- type Metrics
- type MetricsMethodPrometheus
- type MigrateResponse
- type MoreResources
- type MoreResourcesRequest
- type PluginProtoVersion
- type PluginResponse
- type RawResources
- type ResourceBounds
- type ResourceMessage
- type Resources
- func (r Resources) AbsDiff(cmp Resources) Resources
- func (r Resources) CheckValuesAreReasonablySized() error
- func (r Resources) ConvertToRaw(memSlotSize *resource.Quantity) RawResources
- func (r Resources) HasFieldGreaterThan(cmp Resources) bool
- func (r Resources) HasFieldLessThan(cmp Resources) bool
- func (r Resources) IncreaseFrom(old Resources) MoreResources
- func (r Resources) MarshalLogObject(enc zapcore.ObjectEncoder) error
- func (r Resources) Max(cmp Resources) Resources
- func (r Resources) Min(cmp Resources) Resources
- func (r Resources) Mul(factor uint16) Resources
- func (r Resources) ValidateNonZero() error
- type ResumeAgent
- type RunnerProtoVersion
- type ScalingBounds
- type ScalingConfig
- type SuspendAgent
- type UnregisterAgent
- type VCPUCgroup
- type VCPUChange
- type VersionRange
- type VmCpuInfo
- type VmInfo
- type VmMemInfo
Constants ¶
const ( LabelTestingOnlyAlwaysMigrate = "autoscaling.neon.tech/testing-only-always-migrate" LabelEnableAutoscaling = "autoscaling.neon.tech/enabled" AnnotationAutoscalingBounds = "autoscaling.neon.tech/bounds" AnnotationAutoscalingConfig = "autoscaling.neon.tech/config" )
Variables ¶
This section is empty.
Functions ¶
func HasAlwaysMigrateLabel ¶ added in v0.6.0
func HasAlwaysMigrateLabel(obj metav1.ObjectMetaAccessor) bool
func HasAutoscalingEnabled ¶ added in v0.6.0
func HasAutoscalingEnabled(obj metav1.ObjectMetaAccessor) bool
HasAutoscalingEnabled returns true iff the object has the label that enables autoscaling
Types ¶
type AgentDesc ¶
type AgentDesc struct { // AgentID is a unique UUID for the current instance of the autoscaler-agent // // This is helpful so that we can distinguish between (incorrect) duplicate calls to /register // and (correct) re-registering of an agent. AgentID uuid.UUID `json:"agentID"` // ServeAddr gives the unique (per instance) ServerAddr string `json:"agentServeAddr"` // MinProtoVersion is the minimum version of the agent<->informant protocol that the // autoscaler-agent supports // // Protocol versions are always non-zero. // // AgentDesc must always have MinProtoVersion <= MaxProtoVersion. MinProtoVersion InformantProtoVersion `json:"minProtoVersion"` // MaxProtoVersion is the maximum version of the agent<->informant protocol that the // autoscaler-agent supports, inclusive. // // Protocol versions are always non-zero. // // AgentDesc must always have MinProtoVersion <= MaxProtoVersion. MaxProtoVersion InformantProtoVersion `json:"maxProtoVersion"` }
AgentDesc is the first message sent from an autoscaler-agent to a VM informant, describing some information about the autoscaler-agent
Each time an autoscaler-agent (re)connects to a VM informant, it sends an AgentDesc to the "/register" endpoint.
For more information on the agent<->informant protocol, refer to the top-level ARCHITECTURE.md
func (AgentDesc) MarshalLogObject ¶ added in v0.10.0
func (d AgentDesc) MarshalLogObject(enc zapcore.ObjectEncoder) error
MarshalLogObject implements zapcore.ObjectMarshaler, so that Resources can be used with zap.Object
func (AgentDesc) ProtocolRange ¶ added in v0.1.4
func (d AgentDesc) ProtocolRange() VersionRange[InformantProtoVersion]
ProtocolRange returns a VersionRange from d.MinProtoVersion to d.MaxProtoVersion.
type AgentIdentification ¶
type AgentIdentification struct { // AgentID is the same AgentID as given in the AgentDesc initially provided to the VM informant AgentID uuid.UUID `json:"agentID"` }
AgentIdentification affirms the AgentID of the autoscaler-agent in its initial response to a VM informant, on the /id endpoint. This response is always wrapped in an AgentMessage. A type alias for this is provided as AgentIdentificationMessage, for convenience.
type AgentIdentificationMessage ¶
type AgentIdentificationMessage = AgentMessage[AgentIdentification]
type AgentMessage ¶
type AgentMessage[T any] struct { // Data is the content of the request or response Data T `json:"data"` // SequenceNumber is a unique-per-instance monotonically increasing number passed in each // non-initial message from the autoscaler-agent to the VM informant, both requests and // responses. SequenceNumber uint64 `json:"sequenceNumber"` }
AgentMessage is used for (almost) every message sent from the autoscaler-agent to the VM informant, and serves to wrap the type T with a SequenceNumber
The SequenceNumber provides a total ordering of states, even if the ordering of HTTP requests and responses are out of order. Fundamentally this is required because we have bidirectional communication between the autoscaler-agent and VM informant — without it, we run the risk of racy behavior, which could *actually* result in data corruption.
type AgentRequest ¶
type AgentRequest struct { // ProtoVersion is the version of the protocol that the autoscaler-agent is expecting to use // // If the scheduler does not support this version, then it will respond with a 400 status. ProtoVersion PluginProtoVersion `json:"protoVersion"` // Pod is the namespaced name of the pod making the request Pod util.NamespacedName `json:"pod"` // Resources gives a requested or notified change in resources allocated to the VM. // // The requested amount MAY be equal to the current amount, in which case it serves as a // notification that the VM should no longer be contributing to resource pressure. // // TODO: allow passing nil here if nothing's changed (i.e., the request would be the same as the // previous request) Resources Resources `json:"resources"` // Metrics provides information about the VM's current load, so that the scheduler may // prioritize which pods to migrate // // In some protocol versions, this field may be nil. Metrics *Metrics `json:"metrics"` }
AgentRequest is the type of message sent from an autoscaler-agent to the scheduler plugin
All AgentRequests expect a PluginResponse.
func (AgentRequest) ProtocolRange ¶ added in v0.1.8
func (r AgentRequest) ProtocolRange() VersionRange[PluginProtoVersion]
ProtocolRange returns a VersionRange exactly equal to r.ProtoVersion
type AgentResourceMessage ¶ added in v0.10.0
type AgentResourceMessage = AgentMessage[ResourceMessage]
type DownscaleResult ¶
DownscaleResult is used by the VM informant to return whether it downscaled successfully, and some indication of its status when doing so
type InformantDesc ¶
type InformantDesc struct { // ProtoVersion is the version of the agent<->informant protocol that the VM informant has // selected // // If an autoscaler-agent is successfully registered, a well-behaved VM informant MUST respond // with a ProtoVersion within the bounds of the agent's declared minimum and maximum protocol // versions. If the VM informant does not use a protocol version within those bounds, then it // MUST respond with an error status code. ProtoVersion InformantProtoVersion `json:"protoVersion"` // MetricsMethod tells the autoscaler-agent how to fetch metrics from the VM MetricsMethod InformantMetricsMethod `json:"metricsMethod"` }
InformantDesc describes the capabilities of a VM informant, in response to an autoscaler-agent's request on the /register endpoint
For more information on the agent<->informant protocol, refer to the top-level ARCHITECTURE.md
type InformantHealthCheckResp ¶ added in v0.7.0
type InformantHealthCheckResp struct{}
InformantHealthCheckResp is the result of a successful request to a VM informant's /health-check endpoint.
type InformantMetricsMethod ¶
type InformantMetricsMethod struct { // Prometheus describes prometheus-format metrics, typically not through the informant itself Prometheus *MetricsMethodPrometheus `json:"prometheus,omitempty"` }
InformantMetricsMethod collects the options for ways the VM informant can report metrics
At least one method *must* be provided in an InformantDesc, and more than one method gives the autoscaler-agent freedom to choose.
We use this type so it's easier to ensure backwards compatibility with previous versions of the VM informant — at least during the rollout of new autoscaler-agent or VM informant versions.
type InformantProtoVersion ¶ added in v0.1.4
type InformantProtoVersion uint32
InformantProtoVersion represents a single version of the agent<->informant protocol
Each version of the agent<->informant protocol is named independently from releases of the repository containing this code. Names follow semver, although this does not necessarily guarantee support - for example, the VM informant may only support versions above v1.1.
Version compatibility is documented in the neighboring file VERSIONING.md.
const ( // InformantProtoV1_0 represents v1.0 of the agent<->informant protocol - the initial version. // // Last used in release version 0.1.2. InformantProtoV1_0 InformantProtoVersion = iota + 1 // +1 so we start from 1 // InformantProtoV1_1 represents v1.1 of the agent<->informant protocol. // // Changes from v1.0: // // * Adds /try-upscale endpoint to the autoscaler-agent. // // Last used in release version v0.6.0. InformantProtoV1_1 // InformantProtoV1_2 represents v1.2 of the agent<->informant protocol. // // Changes from v1.1: // // * Adds /health-check endpoint to the vm-informant. // // Last used in release version v0.9.0 InformantProtoV1_2 // InformantProtoV2_0 represents v2.0 of the agent<->informant protocol. // // Changes from v1.2: // // * Agents now return a AgentResourceMessage when notifying VM's of changes // in resources on their /upscale and /downscale endpoints. Since // RawResources (the response type in previous protocols) is not // deserializable out of an AgentResourceMessage, this is a breaking // change. // // Currently the latest version. InformantProtoV2_0 )
func (InformantProtoVersion) AllowsHealthCheck ¶ added in v0.7.0
func (v InformantProtoVersion) AllowsHealthCheck() bool
AllowsHealthCheck returns whether this version of the protocol has the informant's /health-check endpoint
This is true for version v1.2 and greater.
func (InformantProtoVersion) HasTryUpscale ¶ added in v0.1.4
func (v InformantProtoVersion) HasTryUpscale() bool
HasTryUpscale returns whether this version of the protocol has the /try-upscale endpoint
This is true for version v1.1 and greater.
func (InformantProtoVersion) IsValid ¶ added in v0.1.4
func (v InformantProtoVersion) IsValid() bool
IsValid returns whether the protocol version is valid. The zero value is not valid.
func (InformantProtoVersion) SignsResourceUpdates ¶ added in v0.10.0
func (v InformantProtoVersion) SignsResourceUpdates() bool
SignsResourceUpdates returns whether agents inform VMs of resource updates with an AgentResourceMessage in this version of the protocol
This is true for version v2.0 and greater
func (InformantProtoVersion) String ¶ added in v0.1.4
func (v InformantProtoVersion) String() string
type Metrics ¶
type Metrics struct { LoadAverage1Min float32 `json:"loadAvg1M"` LoadAverage5Min float32 `json:"loadAvg5M"` MemoryUsageBytes float32 `json:"memoryUsageBytes"` }
Metrics gives the information pulled from node_exporter that the scheduler may use to prioritize which pods it should migrate.
func ReadMetrics ¶
ReadMetrics generates Metrics from node_exporter output, or returns error on failure
This function could be more efficient, but realistically it doesn't matter. The size of the output from node_exporter/vector is so small anyways.
type MetricsMethodPrometheus ¶
type MetricsMethodPrometheus struct {
Port uint16 `json:"port"`
}
MetricsMethodPrometheus describes VM informant's metrics in the prometheus format, made available on a particular port
type MigrateResponse ¶
type MigrateResponse struct{}
MigrateResponse, when provided, is a notification to the autsocaler-agent that it will migrate
After receiving a MigrateResponse, the autoscaler-agent MUST NOT change its resource allocation.
TODO: fill this with more information as required
type MoreResources ¶
type MoreResources struct { // Cpu is true if the VM informant is requesting more CPU Cpu bool `json:"cpu"` // Memory is true if the VM informant is requesting more memory Memory bool `json:"memory"` }
MoreResources holds the data associated with a MoreResourcesRequest
func (MoreResources) And ¶ added in v0.1.4
func (m MoreResources) And(cmp MoreResources) MoreResources
And returns the field-wise logical "and" of m and cmp
func (MoreResources) Not ¶ added in v0.1.4
func (m MoreResources) Not() MoreResources
Not returns the field-wise logical "not" of m
type MoreResourcesRequest ¶ added in v0.1.4
type MoreResourcesRequest struct { MoreResources // ExpectedID is the expected AgentID of the autoscaler-agent ExpectedID uuid.UUID `json:"expectedID"` }
MoreResourcesRequest is the request type wrapping MoreResources that's sent by the VM informant to the autoscaler-agent's /try-upscale endpoint when the VM is urgently in need of more resources.
type PluginProtoVersion ¶ added in v0.1.8
type PluginProtoVersion uint32
PluginProtoVersion represents a single version of the agent<->scheduler plugin protocol
Each version of the agent<->scheduler plugin protocol is named independently from releases of the repository containing this code. Names follow semver, although this does not necessarily guarantee support - for example, the plugin may only support a single version, even though others may appear to be semver-compatible.
Version compatibility is documented in the neighboring file VERSIONING.md.
const ( // PluginProtoV1_0 represents v1.0 of the agent<->scheduler plugin protocol - the initial // version. // // Last used in release version v0.1.8. PluginProtoV1_0 PluginProtoVersion = iota + 1 // start from zero, for backwards compatibility with pre-versioned messages // PluginProtoV1_1 represents v1.1 of the agent<->scheduler plugin protocol. // // Changes from v1.0: // // * Allows a nil value of the AgentRequest.Metrics field. // // Last used in release version v0.6.0. PluginProtoV1_1 // PluginProtoV2_0 represents v2.0 of the agent<->scheduler plugin protocol. // // Changes from v1.1: // // * Supports fractional CPU // // Currently the latest version. PluginProtoV2_0 )
func (PluginProtoVersion) AllowsNilMetrics ¶ added in v0.1.9
func (v PluginProtoVersion) AllowsNilMetrics() bool
AllowsNilMetrics returns whether this version of the protocol allows the autoscaler-agent to send a nil metrics field.
This is true for version v1.1 and greater.
func (PluginProtoVersion) IsValid ¶ added in v0.1.8
func (v PluginProtoVersion) IsValid() bool
IsValid returns whether the protocol version is valid. The zero value is not valid.
func (PluginProtoVersion) String ¶ added in v0.1.8
func (v PluginProtoVersion) String() string
func (PluginProtoVersion) SupportsFractionalCPU ¶ added in v0.7.0
func (v PluginProtoVersion) SupportsFractionalCPU() bool
type PluginResponse ¶
type PluginResponse struct { // Permit provides an upper bound on the resources that the VM is now allowed to consume // // If the request's Resources were less than or equal its current resources, then the Permit // will exactly equal those resources. Otherwise, it may contain resource allocations anywhere // between the current and requested resources, inclusive. Permit Resources `json:"permit"` // Migrate, if present, notifies the autoscaler-agent that its VM will be migrated away, // alongside whatever other information may be useful. Migrate *MigrateResponse `json:"migrate,omitempty"` // ComputeUnit is the minimum unit of resources that the scheduler is expecting to work with // // For example, if ComputeUnit is Resources{ VCPU: 2, Mem: 4 }, then all VMs are expected to // have allocated vCPUs that are a multiple of two, with twice as many memory slots. // // This value may be different across nodes, and the scheduler expects that all AgentRequests // will abide by the most recent ComputeUnit they've received. ComputeUnit Resources `json:"resourceUnit"` }
type RawResources ¶
type RawResources struct { Cpu *resource.Quantity `json:"cpu"` Memory *resource.Quantity `json:"memory"` }
RawResources signals raw resource amounts, and is primarily used in communications with the VM informant because it doesn't know about things like memory slots.
This is used in protocol versions <2. In later versions, AgentResourceMessage is used.
type ResourceBounds ¶ added in v0.7.0
type ResourceMessage ¶ added in v0.10.0
type ResourceMessage struct { RawResources Id AgentIdentification `json:"id"` }
Similar to RawResources, stores raw resource amounts. However, it also stores the ID of the agent notifying the VM of a change in resources. In protocol versions 2 and on, agents notify VM's of changes to their available resources with an AgentResourceMessage. This allows VM informants to verify the authenticity of the agent responding.
type Resources ¶
type Resources struct { VCPU vmapi.MilliCPU `json:"vCPUs"` // Mem gives the number of slots of memory (typically 1G ea.) requested. The slot size is set by // the value of the VM's VirtualMachineSpec.Guest.MemorySlotSize. Mem uint16 `json:"mem"` }
Resources represents an amount of CPU and memory (via memory slots)
When used in an AgentRequest, it represents the desired total amount of resources. When a resource is increasing, the autoscaler-agent "requests" the change to confirm that the resources are available. When decreasing, the autoscaler-agent is expected to use Resources to "notify" the scheduler -- i.e., the resource amount should have already been decreased. When a resource stays at the same amount, the associated AgentRequest serves to indicate that the autoscaler-agent is "satisfied" with its current resources, and should no longer contribute to any existing resource pressure.
When used a PluginResponse (as a Permit), then the Resources serves to inform the autoscaler-agent of the amount it has been permitted to use, subject to node resource limits.
In all cases, each resource type is considered separately from the others.
func (Resources) AbsDiff ¶ added in v0.12.0
AbsDiff returns a new Resources with each field F as the absolute value of the difference between r.F and cmp.F
func (Resources) CheckValuesAreReasonablySized ¶ added in v0.7.0
func (Resources) ConvertToRaw ¶
func (r Resources) ConvertToRaw(memSlotSize *resource.Quantity) RawResources
ConvertToRaw produces the RawResources equivalent to these Resources with the given slot size
func (Resources) HasFieldGreaterThan ¶
HasFieldGreaterThan returns true if and only if there is a field F where r.F > cmp.F
func (Resources) HasFieldLessThan ¶
HasFieldGreaterThan returns true if and only if there is a field F where r.F < cmp.F
func (Resources) IncreaseFrom ¶ added in v0.1.4
func (r Resources) IncreaseFrom(old Resources) MoreResources
Increase returns a MoreResources with each field F true when r.F > old.F.
func (Resources) MarshalLogObject ¶ added in v0.10.0
func (r Resources) MarshalLogObject(enc zapcore.ObjectEncoder) error
MarshalLogObject implements zapcore.ObjectMarshaler, so that Resources can be used with zap.Object
func (Resources) Max ¶
Max returns a new Resources value with each field F as the maximum of r.F and cmp.F
func (Resources) Min ¶
Min returns a new Resources value with each field F as the minimum of r.F and cmp.F
func (Resources) ValidateNonZero ¶
ValidateNonZero checks that neither of the Resources fields are equal to zero, returning an error if either is.
type ResumeAgent ¶
ResumeAgent is sent from the VM informant to the autoscaler-agent to resume contact when it was previously suspended.
type RunnerProtoVersion ¶ added in v0.7.0
type RunnerProtoVersion uint32
this a similar version type for controller <-> runner communications see PluginProtoVersion comment for details
const (
RunnerProtoV1 RunnerProtoVersion = iota + 1
)
func (RunnerProtoVersion) SupportsCgroupFractionalCPU ¶ added in v0.7.0
func (v RunnerProtoVersion) SupportsCgroupFractionalCPU() bool
type ScalingBounds ¶ added in v0.7.0
type ScalingBounds struct { Min ResourceBounds `json:"min"` Max ResourceBounds `json:"max"` }
ScalingBounds is the type that we deserialize from the "autoscaling.neon.tech/bounds" annotation
All fields (and sub-fields) are pointers so that our handling can distinguish between "field not set" and "field equal to zero". Please note that all field are still required to be set and non-zero, though.
type ScalingConfig ¶ added in v0.7.0
type ScalingConfig struct { // LoadAverageFractionTarget sets the desired fraction of current CPU that the load average // should be. For example, with a value of 0.7, we'd want load average to sit at 0.7 × // CPU, // scaling CPU to make this happen. LoadAverageFractionTarget float64 `json:"loadAverageFractionTarget"` // MemoryUsageFractionTarget sets the desired fraction of current memory that // we would like to be using. For example, with a value of 0.7, on a 4GB VM // we'd like to be using 2.8GB of memory. MemoryUsageFractionTarget float64 `json:"memoryUsageFractionTarget"` }
ScalingConfig provides bits of configuration for how the autoscaler-agent makes scaling decisions
func (*ScalingConfig) Validate ¶ added in v0.7.0
func (c *ScalingConfig) Validate() error
type SuspendAgent ¶
SuspendAgent is sent from the VM informant to the autoscaler-agent when it has been contacted by a new autoscaler-agent and wishes to switch to that instead
Instead of just cutting off any connection(s) to the agent, the informant keeps it around in case the new one fails and it needs to fall back to the old one.
type UnregisterAgent ¶
type UnregisterAgent struct { // WasActive indicates whether the unregistered autoscaler-agent was the one in-use by the VM // informant WasActive bool `json:"wasActive"` }
UnregisterAgent is the result of a successful request to a VM informant's /unregister endpoint
type VCPUCgroup ¶ added in v0.7.0
VCPUCgroup is used in runner to reply to controller it represents the vCPU usage as controlled by cgroup
type VCPUChange ¶ added in v0.7.0
VCPUChange is used to notify runner that it had some changes in its CPUs runner uses this info to adjust qemu cgroup
type VersionRange ¶ added in v0.1.4
type VersionRange[V constraints.Ordered] struct { Min V Max V }
VersionRange is a helper type to represent a range of versions.
The bounds are inclusive, representing all versions v with Min <= v <= Max.
func (VersionRange[V]) LatestSharedVersion ¶ added in v0.1.4
func (r VersionRange[V]) LatestSharedVersion(cmp VersionRange[V]) (_ V, ok bool)
LatestSharedVersion returns the latest version covered by both VersionRanges, if there is one.
If either range is invalid, or no such version exists (i.e. the ranges are disjoint), then the returned values will be (0, false).
func (VersionRange[V]) String ¶ added in v0.1.4
func (r VersionRange[V]) String() string
type VmCpuInfo ¶
type VmInfo ¶
type VmInfo struct { Name string `json:"name"` Namespace string `json:"namespace"` Cpu VmCpuInfo `json:"cpu"` Mem VmMemInfo `json:"mem"` ScalingConfig *ScalingConfig `json:"scalingConfig,omitempty"` AlwaysMigrate bool `json:"alwaysMigrate"` ScalingEnabled bool `json:"scalingEnabled"` }
VmInfo is the subset of vmapi.VirtualMachineSpec that the scheduler plugin and autoscaler agent care about. It takes various labels and annotations into account, so certain fields might be different from what's strictly in the VirtualMachine object.
func ExtractVmInfo ¶
func (VmInfo) EqualScalingBounds ¶ added in v0.6.0
func (VmInfo) Format ¶ added in v0.6.0
the reason we have custom formatting for VmInfo is because without it, the formatting of memory slot size (which is a resource.Quantity) has all the fields, meaning (a) it's hard to read, and (b) there's a lot of unnecessary information. But in order to enable "proper" formatting given a VmInfo, we have to implement Format from the top down, so we do.
func (VmInfo) Max ¶
Max returns the Resources representing the maximum amount this VmInfo says the VM may reserve
func (VmInfo) Min ¶
Min returns the Resources representing the minimum amount this VmInfo says the VM must reserve
func (VmInfo) NamespacedName ¶ added in v0.6.0
func (vm VmInfo) NamespacedName() util.NamespacedName