v1alpha1

package
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 24, 2024 License: Apache-2.0 Imports: 4 Imported by: 0

Documentation

Overview

Package v1alpha1 contains API Schema definitions for the operator v1alpha1 API group +kubebuilder:object:generate=true +groupName=gateway-operator.konghq.com

Index

Constants

View Source 
const (
	// AIGatewayConditionTypeAccepted indicates whether or not the controller
	// responsible for this AIGateway has accepted responsibility for the
	// resource and whether active work is underway.
	//
	// Possible reasons for this condition to be "True" include:
	//
	//   - "Accepted"
	//
	// Possible reasons for this condition to be "False" include:
	//
	//   - "Pending"
	//   - "Rejected"
	//
	// "Accepted" is not a terminal condition, if for instance the spec is
	// changed to switch to another (different) GatewayClass cleanup will be
	// performed by the previous GatewayClass and the condition will be set back
	// to "Pending".
	AIGatewayConditionTypeAccepted string = "Accepted"

	// AIGatewayConditionTypeProvisioning indicates whether the controller is
	// actively provisioning the AIGateway, including creating and configuring
	// any necessary resources.
	//
	// Possible reasons for this condition to be "True" include:
	//
	//   - "Deploying"
	//
	// Possible reasons for this condition to be "False" include:
	//
	//   - "Deployed"
	//   - "Failed"
	//
	// If this remains "True" for prolonged periods of time, it may indicate a
	// problem with the controller. Check the most recent status updates and
	// compare them with the controller logs for more details on what may have
	// gone wrong.
	AIGatewayConditionTypeProvisioning string = "Provisioning"

	// AIGatewayConditionTypeEndpointReady indicates whether an endpoint has
	// been provisioned and is ready for inference.
	//
	// Possible reasons for this condition to be "True" include:
	//
	//   - "Deployed"
	//
	// Possible reasons for this condition to be "False" include:
	//
	//   - "Deploying"
	//   - "Failed"
	//
	// While provisioning, some endpoints may be active and ready while others
	// are still being provisioned, or may have failed. Check the AIGateway's
	// status to see the individual status of each endpoint.
	//
	// If this remains "False" for prolonged periods of time, look at the
	// condition's message to determine which endpoints are having trouble.
	// These can include helpful information about what went wrong, e.g.:
	//
	//   - "endpoint foo error: connection refused to https://api.openai.com"
	//
	// You may want to reference the controller logs as well for additional
	// details about what may have gone wrong.
	AIGatewayConditionTypeEndpointReady string = "EndpointReady"
)
View Source 
const (
	// AIGatewayConditionReasonPending indicates that the controller has not yet
	// taken responsibility for the AIGateway.
	//
	// If a resource remains in a prolonged "Pending" state, it may indicate a
	// misconfiguration of the resource (e.g. wrong GatewayClassName) or a
	// problem with the controller. Check the controller logs for more details.
	AIGatewayConditionReasonPending string = "Pending"

	// AIGatewayConditionReasonRejected indicates that the controller has
	// rejected responsibility of the AIGateway.
	//
	// This state requires external intervention to resolve. Check all
	// conditions, condition messages, and perhaps controller logs for help
	// determining the cause.
	AIGatewayConditionReasonRejected string = "Rejected"
)
View Source 
const (
	// AIGatewayConditionReasonDeployed indicates that the controller has
	// completed provisioning.
	//
	// This is not a terminal condition, updates to the specification may
	// trigger new provisioning work.
	AIGatewayConditionReasonDeployed string = "Deployed"

	// AIEndpointConditionReasonFailed indicates that the controller has
	// failed to provision the AIGateway.
	//
	// This is a terminal condition and requires external intervention to
	// resolve. Check all conditions, condition messages, and perhaps controller
	// logs for help finding the cause.
	AIGatewayConditionReasonFailed string = "Failed"
)
View Source 
const (
	// AIGatewayConditionReasonAccepted indicates that the controller has
	// accepted responsibility for the AIGateway.
	AIGatewayConditionReasonAccepted string = "Accepted"
)
View Source 
const (
	// AIGatewayConditionReasonDeploying indicates that the controller is
	// actively working on deployment.
	//
	// This is a transient condition and should not remain "True" for prolonged
	// periods of time.
	//
	// If this condition remains "True" for a prolonged period of time, check
	// all conditions, condition messages, and perhaps controller logs for help
	// determining the cause.
	AIGatewayConditionReasonDeploying string = "Deploying"
)
View Source 
const (
	// DataPlaneMetricsExtensionKind holds the kind for the DataPlaneMetricsExtension.
	DataPlaneMetricsExtensionKind = "DataPlaneMetricsExtension"
)

Variables

View Source 
var (
	// SchemeGroupVersion is group version used to register these objects
	SchemeGroupVersion = schema.GroupVersion{Group: "gateway-operator.konghq.com", Version: "v1alpha1"}

	// SchemeBuilder is used to add go types to the GroupVersionKind scheme
	SchemeBuilder = &scheme.Builder{GroupVersion: SchemeGroupVersion}

	// AddToScheme adds the types in this group-version to the given scheme.
	AddToScheme = SchemeBuilder.AddToScheme
)

Functions

func AIGatewayGVR added in v1.3.0 

func AIGatewayGVR() schema.GroupVersionResource

AIGatewayGVR returns current package AIGateway GVR.

Types

type AICloudProvider 

type AICloudProvider struct {
	// Name is the unique name of an LLM provider.
	//
	// +kubebuilder:validation:Required
	// +kubebuilder:validation:Enum=openai;azure;cohere;mistral
	Name AICloudProviderName `json:"name"`
}

AICloudProvider is the organization that provides API access to Large Language Models (LLMs).

func (*AICloudProvider) DeepCopy 

func (in *AICloudProvider) DeepCopy() *AICloudProvider

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AICloudProvider.

func (*AICloudProvider) DeepCopyInto 

func (in *AICloudProvider) DeepCopyInto(out *AICloudProvider)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type AICloudProviderAPITokenRef 

type AICloudProviderAPITokenRef struct {
	// Name is the name of the reference object.
	//
	// +kubebuilder:validation:Required
	Name string `json:"name"`

	// Namespace is the namespace of the reference object.
	//
	// If not specified, it will be assumed to be the same namespace as the
	// object which references it.
	//
	// +kubebuilder:validation:Optional
	Namespace *string `json:"namespace,omitempty"`

	// Kind is the API object kind
	//
	// If not specified, it will be assumed to be "Secret". If a Secret is used
	// as the Kind, the secret must contain a single key-value pair where the
	// value is the secret API token. The key can be named anything, as long as
	// there's only one entry, but by convention it should be "apiToken".
	//
	// +kubebuilder:validation:Optional
	Kind *string `json:"kind,omitempty"`
}

AICloudProviderAPITokenRef is an reference to another object which contains the API token for an AI cloud provider.

func (*AICloudProviderAPITokenRef) DeepCopy 

func (in *AICloudProviderAPITokenRef) DeepCopy() *AICloudProviderAPITokenRef

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AICloudProviderAPITokenRef.

func (*AICloudProviderAPITokenRef) DeepCopyInto 

func (in *AICloudProviderAPITokenRef) DeepCopyInto(out *AICloudProviderAPITokenRef)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type AICloudProviderName 

type AICloudProviderName string

AICloudProviderName indicates the unique name of a supported AI cloud provider. 

const (
	// AICloudProviderOpenAI is the OpenAI cloud provider.
	//
	// They are known for models such as ChatGPT 3.5, 4, Dall-e, e.t.c.
	AICloudProviderOpenAI AICloudProviderName = "openai"

	// AICloudProviderAzure is the Azure cloud provider.
	//
	// They are known for models such as PHI-2.
	AICloudProviderAzure AICloudProviderName = "azure"

	// AICloudProviderCohere is the Cohere cloud provider.
	//
	// They are known for models such as Cohere-Embed, and Cohere-Rerank.
	AICloudProviderCohere AICloudProviderName = "cohere"

	// AICloudProviderMistral is the Mistral.AI cloud provider.
	//
	// They are known for models such as mistral-tiny.
	AICloudProviderMistral AICloudProviderName = "mistral"
)

type AIGateway 

type AIGateway struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	// Spec is the desired state of the AIGateway.
	//
	// +kubebuilder:validation:XValidation:message="At least one type of LLM has been specified",rule="(self.largeLanguageModels != null)"
	Spec AIGatewaySpec `json:"spec,omitempty"`

	// Status is the observed state of the AIGateway.
	Status AIGatewayStatus `json:"status,omitempty"`
}

AIGateway is a network Gateway enabling access and management for AI & Machine Learning models such as Large Language Models (LLM).

The underlying technology for the AIGateway is the Kong Gateway configured with a variety of plugins which provide the the AI featureset.

This is a list of the plugins, which are available in Kong Gateway v3.6.x+:

So effectively the AIGateway resource provides a bespoke Gateway resource (which it owns and manages) with the gateway, consumers and plugin configurations automated and configurable via Kubernetes APIs.

The current iteration only supports the proxy itself, but the API is being built with room for future growth in several dimensions. For instance:

  • Supporting auxiliary functions (e.g. decorator, guard, templater, token-rate-limit)
  • Supporting request/response transformers
  • Supporting more than just LLMs (e.g. CCNs, GANs, e.t.c.)
  • Supporting more hosting options for LLMs (e.g. self hosted)
  • Supporting more AI cloud providers
  • Supporting more AI cloud provider features

The validation rules throughout are set up to ensure at least one cloud-provider-based LLM is specified, but in the future when we have more model types and more hosting options for those types so we may want to look into using CEL validation to ensure that at least one model configuration is provided. We may also want to use CEL to validate things like identifier unique-ness, e.t.c.

See: https://kubernetes.io/docs/reference/using-api/cel/

+genclient +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object +kubebuilder:object:root=true +kubebuilder:subresource:status +kubebuilder:printcolumn:name="Endpoint",type="string",JSONPath=".status.endpoint",description="The URL endpoint for the AIGateway"

func (*AIGateway) DeepCopy 

func (in *AIGateway) DeepCopy() *AIGateway

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AIGateway.

func (*AIGateway) DeepCopyInto 

func (in *AIGateway) DeepCopyInto(out *AIGateway)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*AIGateway) DeepCopyObject 

func (in *AIGateway) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

func (*AIGateway) GetConditions 

func (a *AIGateway) GetConditions() []metav1.Condition

GetConditions returns the status conditions.

func (*AIGateway) SetConditions 

func (a *AIGateway) SetConditions(conditions []metav1.Condition)

SetConditions sets the status conditions.

type AIGatewayConsumerRef 

type AIGatewayConsumerRef struct {
	// Name is the name of the reference object.
	//
	// +kubebuilder:validation:Required
	Name string `json:"name"`

	// Namespace is the namespace of the reference object.
	//
	// +kubebuilder:validation:Required
	Namespace string `json:"namespace"`
}

AIGatewayConsumerRef indicates the Secret resource containing the credentials for the Kong consumer.

func (*AIGatewayConsumerRef) DeepCopy 

func (in *AIGatewayConsumerRef) DeepCopy() *AIGatewayConsumerRef

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AIGatewayConsumerRef.

func (*AIGatewayConsumerRef) DeepCopyInto 

func (in *AIGatewayConsumerRef) DeepCopyInto(out *AIGatewayConsumerRef)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type AIGatewayEndpoint 

type AIGatewayEndpoint struct {
	// NetworkAccessHint is a hint to the user about what kind of network access
	// is expected for the reachability of this endpoint.
	NetworkAccessHint EndpointNetworkAccessHint `json:"network"`

	// URL is the URL to access the endpoint from the network indicated by the
	// NetworkAccessHint.
	URL string `json:"url"`

	// AvailableModels is a list of the identifiers of all the AI models that are
	// accessible from this endpoint.
	AvailableModels []string `json:"models"`

	// Consumer is a reference to the Secret that contains the credentials for
	// the Kong consumer that is allowed to access this endpoint.
	Consumer AIGatewayConsumerRef `json:"consumer"`

	// Conditions describe the current conditions of the AIGatewayEndpoint.
	//
	// Known condition types are:
	//
	//   - "Provisioning"
	//   - "EndpointReady"
	//
	// +listType=map
	// +listMapKey=type
	// +kubebuilder:validation:Optional
	// +kubebuilder:validation:MaxItems=8
	// +kubebuilder:default={{type: "Provisioning", status: "Unknown", reason:"Pending", message:"Waiting for controller", lastTransitionTime: "1970-01-01T00:00:00Z"}}
	Conditions []metav1.Condition `json:"conditions,omitempty"`
}

AIGatewayEndpoint is a network endpoint for accessing an AIGateway.

func (*AIGatewayEndpoint) DeepCopy 

func (in *AIGatewayEndpoint) DeepCopy() *AIGatewayEndpoint

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AIGatewayEndpoint.

func (*AIGatewayEndpoint) DeepCopyInto 

func (in *AIGatewayEndpoint) DeepCopyInto(out *AIGatewayEndpoint)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type AIGatewayList 

type AIGatewayList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`

	// Items is the list of AIGateways.
	Items []AIGateway `json:"items"`
}

AIGatewayList contains a list of AIGateways.

+kubebuilder:object:root=true

func (*AIGatewayList) DeepCopy 

func (in *AIGatewayList) DeepCopy() *AIGatewayList

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AIGatewayList.

func (*AIGatewayList) DeepCopyInto 

func (in *AIGatewayList) DeepCopyInto(out *AIGatewayList)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*AIGatewayList) DeepCopyObject 

func (in *AIGatewayList) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

type AIGatewaySpec 

type AIGatewaySpec struct {
	// GatewayClassName is the name of the GatewayClass which is responsible for
	// the AIGateway.
	//
	// +kubebuilder:validation:Required
	GatewayClassName string `json:"gatewayClassName"`

	// LargeLanguageModels is a list of Large Language Models (LLMs) to be
	// managed by the AI Gateway.
	//
	// This is a required field because we only support LLMs at the moment. In
	// future iterations we may support other model types.
	//
	// +kubebuilder:validation:Required
	// +kubebuilder:validation:XValidation:message="At least one class of LLMs has been configured",rule="(self.cloudHosted.size() != 0)"
	LargeLanguageModels *LargeLanguageModels `json:"largeLanguageModels,omitempty"`

	// CloudProviderCredentials is a reference to an object (e.g. a Kubernetes
	// Secret) which contains the credentials needed to access the APIs of
	// cloud providers.
	//
	// This is the global configuration that will be used by DEFAULT for all
	// model configurations. A secret configured this way MAY include any number
	// of key-value pairs equal to the number of providers you have, but used
	// this way the keys MUST be named according to their providers (e.g.
	// "openai", "azure", "cohere", e.t.c.). For example:
	//
	//   apiVersion: v1
	//   kind: Secret
	//   metadata:
	//     name: devteam-ai-cloud-providers
	//   type: Opaque
	//   data:
	//     openai: *****************
	//     azure: *****************
	//     cohere: *****************
	//
	// See AICloudProviderName for a list of known and valid cloud providers.
	//
	// Note that the keys are NOT case-sensitive (e.g. "OpenAI", "openai", and
	// "openAI" are all valid and considered the same keys) but if there are
	// duplicates endpoints failures conditions will be emitted and endpoints
	// will not be configured until the duplicates are resolved.
	//
	// This is currently considered required, but in future iterations will be
	// optional as we do things like enable configuring credentials at the model
	// level.
	//
	// +kubebuilder:validation:Required
	CloudProviderCredentials *AICloudProviderAPITokenRef `json:"cloudProviderCredentials,omitempty"`
}

AIGatewaySpec defines the desired state of an AIGateway.

func (*AIGatewaySpec) DeepCopy 

func (in *AIGatewaySpec) DeepCopy() *AIGatewaySpec

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AIGatewaySpec.

func (*AIGatewaySpec) DeepCopyInto 

func (in *AIGatewaySpec) DeepCopyInto(out *AIGatewaySpec)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type AIGatewayStatus 

type AIGatewayStatus struct {
	// Endpoints are collections of the URL, credentials and metadata needed in
	// order to access models served by the AIGateway for inference.
	//
	// +kubebuilder:validation:Optional
	// +kubebuilder:validation:MaxItems=64
	Endpoints []AIGatewayEndpoint `json:"endpoints,omitempty"`

	// Conditions describe the current conditions of the AIGateway.
	//
	// Known condition types are:
	//
	//   - "Accepted"
	//   - "Provisioning"
	//   - "EndpointsReady"
	//
	// +listType=map
	// +listMapKey=type
	// +kubebuilder:validation:Optional
	// +kubebuilder:validation:MaxItems=8
	// +kubebuilder:default={{type: "Accepted", status: "Unknown", reason:"Pending", message:"Waiting for controller", lastTransitionTime: "1970-01-01T00:00:00Z"}}
	Conditions []metav1.Condition `json:"conditions,omitempty"`
}

AIGatewayStatus defines the observed state of AIGateway.

func (*AIGatewayStatus) DeepCopy 

func (in *AIGatewayStatus) DeepCopy() *AIGatewayStatus

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AIGatewayStatus.

func (*AIGatewayStatus) DeepCopyInto 

func (in *AIGatewayStatus) DeepCopyInto(out *AIGatewayStatus)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type CloudHostedLargeLanguageModel 

type CloudHostedLargeLanguageModel struct {
	// Identifier is the unique name which identifies the LLM. This will be used
	// as part of the requests made to an AIGateway endpoint. For instance: if
	// you provided the identifier "devteam-gpt-access", then you would access
	// this model via "https://${endpoint}/devteam-gpt-access" and supply it
	// with your consumer credentials to authenticate requests.
	//
	// +kubebuilder:validation:Required
	Identifier string `json:"identifier"`

	// Model is the model name of the LLM (e.g. gpt-3.5-turbo, phi-2, e.t.c.).
	//
	// If not specified, whatever the cloud provider specifies as the default
	// model will be used.
	//
	// +kubebuilder:validation:Optional
	Model *string `json:"model"`

	// PromptType is the type of prompt to be used for inference requests to
	// the LLM (e.g. "chat", "completions").
	//
	// If "chat" is specified, prompts sent by the user will be interactive,
	// contextual and stateful. The LLM will dynamically answer questions and
	// simulate a dialogue, while also keeping track of the conversation to
	// provide contextually relevant responses.
	//
	// If "completions" is specified, prompts sent by the user will be
	// stateless and "one-shot". The LLM will provide a single response to the
	// prompt, without any context from previous prompts.
	//
	// If not specified, "completions" will be used as the default.
	//
	// +kubebuilder:validation:Optional
	// +kubebuilder:validation:Enum=chat;completions
	// +kubebuilder:default=completions
	PromptType *LLMPromptType `json:"promptType"`

	// DefaultPrompts is a list of prompts that should be provided to the LLM
	// by default. This is generally used to influence inference behavior, for
	// instance by providing a "system" role prompt that instructs the LLM to
	// take on a certain persona.
	//
	// +kubebuilder:validation:Optional
	// +kubebuilder:validation:MaxItems=64
	DefaultPrompts []LLMPrompt `json:"defaultPrompts"`

	// DefaultPromptParams configures the parameters which will be sent with
	// any and every inference request.
	//
	// If this is set, there is currently no way to override these parameters
	// at the individual prompt level. This is an expected feature from later
	// releases of our AI plugins.
	//
	// +kubebuilder:validation:Optional
	DefaultPromptParams *LLMPromptParams `json:"defaultPromptParams"`

	// AICloudProvider defines the cloud provider that will fulfill the LLM
	// requests for this CloudHostedLargeLanguageModel
	//
	// +kubebuilder:validation:Required
	AICloudProvider AICloudProvider `json:"aiCloudProvider"`
}

CloudHostedLargeLanguageModel is the configuration for Large Language Models (LLM) hosted by a known and supported AI cloud provider (e.g. OpenAI, Cohere, Azure, e.t.c.).

func (*CloudHostedLargeLanguageModel) DeepCopy 

func (in *CloudHostedLargeLanguageModel) DeepCopy() *CloudHostedLargeLanguageModel

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CloudHostedLargeLanguageModel.

func (*CloudHostedLargeLanguageModel) DeepCopyInto 

func (in *CloudHostedLargeLanguageModel) DeepCopyInto(out *CloudHostedLargeLanguageModel)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type DataPlaneMetricsExtension 

type DataPlaneMetricsExtension struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   DataPlaneMetricsExtensionSpec   `json:"spec,omitempty"`
	Status DataPlaneMetricsExtensionStatus `json:"status,omitempty"`
}

DataPlaneMetricsExtension holds the configuration for the DataPlane metrics extension. It can be attached to a ControlPlane using its spec.extensions. When attached it will make the ControlPlane configure its DataPlane with the specified metrics configuration. Additionally, it will also make the operator expose DataPlane's metrics enriched with metadata required for in-cluster Kubernetes autoscaling.

NOTE: This is an enterprise feature. In order to use it you need to use the EE version of Kong Gateway Operator with a valid license.

func (*DataPlaneMetricsExtension) DeepCopy 

func (in *DataPlaneMetricsExtension) DeepCopy() *DataPlaneMetricsExtension

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataPlaneMetricsExtension.

func (*DataPlaneMetricsExtension) DeepCopyInto 

func (in *DataPlaneMetricsExtension) DeepCopyInto(out *DataPlaneMetricsExtension)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*DataPlaneMetricsExtension) DeepCopyObject 

func (in *DataPlaneMetricsExtension) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

type DataPlaneMetricsExtensionList 

type DataPlaneMetricsExtensionList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []DataPlaneMetricsExtension `json:"items"`
}

DataPlaneMetricsExtensionList contains a list of DataPlaneMetricsExtension.

func (*DataPlaneMetricsExtensionList) DeepCopy 

func (in *DataPlaneMetricsExtensionList) DeepCopy() *DataPlaneMetricsExtensionList

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataPlaneMetricsExtensionList.

func (*DataPlaneMetricsExtensionList) DeepCopyInto 

func (in *DataPlaneMetricsExtensionList) DeepCopyInto(out *DataPlaneMetricsExtensionList)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*DataPlaneMetricsExtensionList) DeepCopyObject 

func (in *DataPlaneMetricsExtensionList) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

type DataPlaneMetricsExtensionSpec 

type DataPlaneMetricsExtensionSpec struct {
	// ServiceSelector holds the service selector specifying the services
	// for which metrics should be collected.
	//
	// +kubebuilder:validation:Required
	ServiceSelector ServiceSelector `json:"serviceSelector"`

	// Config holds the configuration for the DataPlane metrics.
	//
	// +kube:validation:Optional
	Config MetricsConfig `json:"config,omitempty"`
}

DataPlaneMetricsExtensionSpec defines the spec for the DataPlaneMetricsExtension.

func (*DataPlaneMetricsExtensionSpec) DeepCopy 

func (in *DataPlaneMetricsExtensionSpec) DeepCopy() *DataPlaneMetricsExtensionSpec

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataPlaneMetricsExtensionSpec.

func (*DataPlaneMetricsExtensionSpec) DeepCopyInto 

func (in *DataPlaneMetricsExtensionSpec) DeepCopyInto(out *DataPlaneMetricsExtensionSpec)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type DataPlaneMetricsExtensionStatus 

type DataPlaneMetricsExtensionStatus struct {
	// ControlPlaneRef is a reference to the ControlPlane that this is associated with.
	// This field is set by the operator when this extension is associated with
	// a ControlPlane through its extensions spec.
	// There can only be one ControlPlane associated with a given DataPlaneMetricsExtension.
	// When this is unset it means that the association has been removed.
	//
	// +kube:validation:Optional
	ControlPlaneRef *NamespacedRef `json:"controlPlaneRef,omitempty"`
}

DataPlaneMetricsExtensionStatus defines the status of the DataPlaneMetricsExtension.

func (*DataPlaneMetricsExtensionStatus) DeepCopy 

func (in *DataPlaneMetricsExtensionStatus) DeepCopy() *DataPlaneMetricsExtensionStatus

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataPlaneMetricsExtensionStatus.

func (*DataPlaneMetricsExtensionStatus) DeepCopyInto 

func (in *DataPlaneMetricsExtensionStatus) DeepCopyInto(out *DataPlaneMetricsExtensionStatus)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type EndpointNetworkAccessHint 

type EndpointNetworkAccessHint string

EndpointNetworkAccessHint provides a human readable indication of what kind of network access is expected for a Gateway.

This isn't meant to reflect knowledge of any specific network by name, which is why it includes "hint" in the name. It's meant to be a hint to the user such as "internet-accessible", "internal-only". 

const (
	// NetworkInternetAccessible indicates that the endpoint is accessible from
	// the public internet.
	NetworkInternetAccessible EndpointNetworkAccessHint = "internet-accessible"
)

type ExtensionRef 

type ExtensionRef struct {
	// Group is the group of the extension resource.
	Group string `json:"group"`

	// Kind is kind of the extension resource.
	Kind string `json:"kind"`

	// NamespacedRef is a reference to the extension resource.
	NamespacedRef `json:",inline"`
}

ExtensionRef corresponds to another resource in the Kubernetes cluster which defines extended behavior for a resource (e.g. ControlPlane).

func (*ExtensionRef) DeepCopy 

func (in *ExtensionRef) DeepCopy() *ExtensionRef

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ExtensionRef.

func (*ExtensionRef) DeepCopyInto 

func (in *ExtensionRef) DeepCopyInto(out *ExtensionRef)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type LLMPrompt 

type LLMPrompt struct {
	// Content is the prompt text sent for inference.
	//
	// +kubebuilder:validation:Required
	Content string `json:"content"`

	// Role indicates the role of the prompt. This is used to identify the
	// prompt's purpose, such as "system" or "user" and can influence the
	// behavior of the LLM.
	//
	// If not specified, "user" will be used as the default.
	//
	// +kubebuilder:validation:Optional
	// +kubebuilder:validation:Enum=user;system
	// +kubebuilder:default=user
	Role *LLMPromptRole `json:"role"`
}

LLMPrompt is a text prompt that includes parameters, a role and content.

This is intended for situations like when you need to provide roles in a prompt to an LLM in order to influence its behavior and responses.

For example, you might want to provide a "system" role and tell the LLM something like "you are a helpful assistant who responds in the style of Sherlock Holmes".

func (*LLMPrompt) DeepCopy 

func (in *LLMPrompt) DeepCopy() *LLMPrompt

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LLMPrompt.

func (*LLMPrompt) DeepCopyInto 

func (in *LLMPrompt) DeepCopyInto(out *LLMPrompt)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type LLMPromptParams 

type LLMPromptParams struct {
	// Temperature controls the randomness of predictions by scaling the logits
	// before applying softmax. A lower temperature (e.g., 0.0 to 0.7) makes
	// the model more confident in its predictions, leading to more repetitive
	// and deterministic outputs. A higher temperature (e.g., 0.8 to 1.0)
	// increases randomness, generating more diverse and creative outputs. At
	// very high temperatures, the outputs may become nonsensical or highly
	// unpredictable.
	//
	// +kubebuilder:validation:Optional
	Temperature *string `json:"temperature"`

	// Max Tokens specifies the maximum length of the model's output in terms
	// of the number of tokens (words or pieces of words). This parameter
	// limits the output's size, ensuring the model generates content within a
	// manageable scope. A token can be a word or part of a word, depending on
	// the model's tokenizer.
	//
	// +kubebuilder:validation:Optional
	MaxTokens *int `json:"maxTokens"`

	// TopK sampling is a technique where the model's prediction is limited to
	// the K most likely next tokens at each step of the generation process.
	// The probability distribution is truncated to these top K tokens, and the
	// next token is randomly sampled from this subset. This method helps in
	// reducing the chance of selecting highly improbable tokens, making the
	// text more coherent. A smaller K leads to more predictable text, while a
	// larger K allows for more diversity but with an increased risk of
	// incoherence.
	//
	// +kubebuilder:validation:Optional
	TopK *int `json:"topK"`

	// TopP (also known as nucleus sampling) is an alternative to top K
	// sampling. Instead of selecting the top K tokens, top P sampling chooses
	// from the smallest set of tokens whose cumulative probability exceeds the
	// threshold P. This method dynamically adjusts the number of tokens
	// considered at each step, depending on their probability distribution. It
	// helps in maintaining diversity while also avoiding very unlikely tokens.
	// A higher P value increases diversity but can lead to less coherence,
	// whereas a lower P value makes the model's outputs more focused and
	// coherent.
	//
	// +kubebuilder:validation:Optional
	TopP *string `json:"topP"`
}

LLMPromptParams contains parameters that can be used to control the behavior of a large language model (LLM) when generating text based on a prompt.

func (*LLMPromptParams) DeepCopy 

func (in *LLMPromptParams) DeepCopy() *LLMPromptParams

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LLMPromptParams.

func (*LLMPromptParams) DeepCopyInto 

func (in *LLMPromptParams) DeepCopyInto(out *LLMPromptParams)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type LLMPromptRole 

type LLMPromptRole string

LLMPromptRole indicates the role of a prompt for a large language model (LLM). 

const (
	// LLMPromptRoleUser indicates that the prompt is for the user.
	LLMPromptRoleUser LLMPromptRole = "user"

	// LLMPromptRoleSystem indicates that the prompt is for the system.
	LLMPromptRoleSystem LLMPromptRole = "system"

	// LLMPromptRoleAssistant indicates that the prompt is for the 'virtual assistant'.
	// It represents something that the chat bot "did", or "theoretically could have," said.
	LLMPromptRoleAssistant LLMPromptRole = "assistance"
)

type LLMPromptType 

type LLMPromptType string

LLMPromptType indicates the type of prompt to be used for a large language model (LLM). 

const (
	// LLMPromptTypeChat indicates that the prompt is for a chat.
	LLMPromptTypeChat LLMPromptType = "chat"

	// LLMPromptTypeCompletion indicates that the prompt is for a completion.
	LLMPromptTypeCompletion LLMPromptType = "completions"
)

type LargeLanguageModels 

type LargeLanguageModels struct {
	// CloudHosted configures LLMs hosted and served by cloud providers.
	//
	// This is currently a required field, requiring at least one cloud-hosted
	// LLM be specified, however in future iterations we may add other hosting
	// options such as self-hosted LLMs as separate fields.
	//
	// +kubebuilder:validation:Required
	// +kubebuilder:validation:MinItems=1
	// +kubebuilder:validation:MaxItems=64
	CloudHosted []CloudHostedLargeLanguageModel `json:"cloudHosted"`
}

LargeLanguageModels is a list of Large Language Models (LLM) hosted in various ways (cloud hosted, self hosted, e.t.c.) which the AIGateway should serve and manage traffic for.

func (*LargeLanguageModels) DeepCopy 

func (in *LargeLanguageModels) DeepCopy() *LargeLanguageModels

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LargeLanguageModels.

func (*LargeLanguageModels) DeepCopyInto 

func (in *LargeLanguageModels) DeepCopyInto(out *LargeLanguageModels)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type MetricsConfig 

type MetricsConfig struct {
	// Latency indicates whether latency metrics are enabled for the DataPlane.
	// This translates into deployed instances having `latency_metrics` option set
	// on the Prometheus plugin.
	//
	// +kubebuilder:default=false
	// +kube:validation:Optional
	Latency bool `json:"latency"`

	// Bandwidth indicates whether bandwidth metrics are enabled for the DataPlane.
	// This translates into deployed instances having `bandwidth_metrics` option set
	// on the Prometheus plugin.
	//
	// +kubebuilder:default=false
	// +kube:validation:Optional
	Bandwidth bool `json:"bandwidth"`

	// UpstreamHealth indicates whether upstream health metrics are enabled for the DataPlane.
	// This translates into deployed instances having `upstream_health_metrics` option set
	// on the Prometheus plugin.
	//
	// +kubebuilder:default=false
	// +kube:validation:Optional
	UpstreamHealth bool `json:"upstreamHealth"`

	// StatusCode indicates whether status code metrics are enabled for the DataPlane.
	// This translates into deployed instances having `status_code_metrics` option set
	// on the Prometheus plugin.
	//
	// +kubebuilder:default=false
	// +kube:validation:Optional
	StatusCode bool `json:"statusCode"`
}

MetricsConfig holds the configuration for the DataPlane metrics.

func (*MetricsConfig) DeepCopy 

func (in *MetricsConfig) DeepCopy() *MetricsConfig

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new MetricsConfig.

func (*MetricsConfig) DeepCopyInto 

func (in *MetricsConfig) DeepCopyInto(out *MetricsConfig)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type NamespacedRef 

type NamespacedRef struct {
	// Name is the name of the referred resource.
	//
	// +kubebuilder:validation:MinLength=1
	// +kubebuilder:validation:MaxLength=253
	Name string `json:"name"`

	// Namespace is the namespace of the referred resource.
	//
	// For namespace-scoped resources if no Namespace is provided then the
	// namespace of the parent object MUST be used.
	//
	// This field MUST not be set when referring to cluster-scoped resources.
	//
	// +optional
	Namespace *string `json:"namespace,omitempty"`
}

NamespacedRef is a reference to a namespaced resource.

func (*NamespacedRef) DeepCopy 

func (in *NamespacedRef) DeepCopy() *NamespacedRef

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new NamespacedRef.

func (*NamespacedRef) DeepCopyInto 

func (in *NamespacedRef) DeepCopyInto(out *NamespacedRef)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type ServiceSelector 

type ServiceSelector struct {
	// MatchNames holds the list of Services names to match.
	//
	// +listType=map
	// +listMapKey=name
	// +kubebuilder:validation:Required
	MatchNames []ServiceSelectorEntry `json:"matchNames,omitempty"`
}

ServiceSelector holds the service selector specification.

func (*ServiceSelector) DeepCopy 

func (in *ServiceSelector) DeepCopy() *ServiceSelector

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ServiceSelector.

func (*ServiceSelector) DeepCopyInto 

func (in *ServiceSelector) DeepCopyInto(out *ServiceSelector)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type ServiceSelectorEntry 

type ServiceSelectorEntry struct {
	// Name is the name of the service to match.
	//
	// +kubebuilder:validation:Required
	Name string `json:"name"`
}

ServiceSelectorEntry holds the name of a service to match.

func (*ServiceSelectorEntry) DeepCopy 

func (in *ServiceSelectorEntry) DeepCopy() *ServiceSelectorEntry

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ServiceSelectorEntry.

func (*ServiceSelectorEntry) DeepCopyInto 

func (in *ServiceSelectorEntry) DeepCopyInto(out *ServiceSelectorEntry)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.