v1alpha1

Documentation

Overview

Package v1alpha1 contains API Schema definitions for the serving v1alpha1 API group +kubebuilder:object:generate=true +groupName=serving.kubedl.io

Index

• Constants
• Variables
• func RegisterDefaults(scheme *runtime.Scheme) error
• func SetDefaults_Inference(in *Inference)
• func SetObjectDefaults_Inference(in *Inference)
• type AutoScaleStrategy
  • func (in *AutoScaleStrategy) DeepCopy() *AutoScaleStrategy
  • func (in *AutoScaleStrategy) DeepCopyInto(out *AutoScaleStrategy)
• type BatchingStrategy
  • func (in *BatchingStrategy) DeepCopy() *BatchingStrategy
  • func (in *BatchingStrategy) DeepCopyInto(out *BatchingStrategy)
• type FrameworkType
• type Inference
  • func (in *Inference) DeepCopy() *Inference
  • func (in *Inference) DeepCopyInto(out *Inference)
  • func (in *Inference) DeepCopyObject() runtime.Object
• type InferenceList
  • func (in *InferenceList) DeepCopy() *InferenceList
  • func (in *InferenceList) DeepCopyInto(out *InferenceList)
  • func (in *InferenceList) DeepCopyObject() runtime.Object
• type InferenceSpec
  • func (in *InferenceSpec) DeepCopy() *InferenceSpec
  • func (in *InferenceSpec) DeepCopyInto(out *InferenceSpec)
• type InferenceStatus
  • func (in *InferenceStatus) DeepCopy() *InferenceStatus
  • func (in *InferenceStatus) DeepCopyInto(out *InferenceStatus)
• type PredictorSpec
  • func (in *PredictorSpec) DeepCopy() *PredictorSpec
  • func (in *PredictorSpec) DeepCopyInto(out *PredictorSpec)
• type PredictorStatus
  • func (in *PredictorStatus) DeepCopy() *PredictorStatus
  • func (in *PredictorStatus) DeepCopyInto(out *PredictorStatus)

Constants

const (
	DefaultPredictorContainerName         = "predictor"
	DefaultPredictorContainerHttpPortName = "predictor-http-port"
	DefaultPredictorContainerGrpcPortName = "predictor-grpc-port"
	DefaultPredictorContainerHttpPort     = 8080
	DefaultPredictorContainerGrpcPort     = 9000
	DefaultModelMountPath                 = "/mnt/models"
)

Default constants values for inference service.

Variables

var (
	// GroupVersion is group version used to register these objects
	GroupVersion = schema.GroupVersion{Group: "serving.kubedl.io", Version: "v1alpha1"}

	// SchemeBuilder is used to add go types to the GroupVersionKind scheme
	SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion}

	// AddToScheme adds the types in this group-version to the given scheme.
	AddToScheme = SchemeBuilder.AddToScheme
)

Functions

func RegisterDefaults

func RegisterDefaults(scheme *runtime.Scheme) error

RegisterDefaults adds defaulters functions to the given scheme. Public to allow building arbitrary schemes. All generated defaulters are covering - they call all nested defaulters.

func SetDefaults_Inference

func SetDefaults_Inference(in *Inference)

func SetObjectDefaults_Inference

func SetObjectDefaults_Inference(in *Inference)

Types

type AutoScaleStrategy

type AutoScaleStrategy struct {
	MinReplicas *int32                 `json:"minReplicas,omitempty"`
	MaxReplicas *int32                 `json:"maxReplicas,omitempty"`
	AutoScaler  corev1.ObjectReference `json:"autoScaler"`
}

func (*AutoScaleStrategy) DeepCopy

func (in *AutoScaleStrategy) DeepCopy() *AutoScaleStrategy

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutoScaleStrategy.

func (*AutoScaleStrategy) DeepCopyInto

func (in *AutoScaleStrategy) DeepCopyInto(out *AutoScaleStrategy)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type BatchingStrategy

type BatchingStrategy struct {
	// BatchSize represents batch size of requests to be forwarded to backend
	// service.
	BatchSize int32 `json:"batchSize"`
	// TimeoutSeconds represents max waiting duration to aggregate batch requests.
	TimeoutSeconds *int32 `json:"timeoutSeconds,omitempty"`
}

func (*BatchingStrategy) DeepCopy

func (in *BatchingStrategy) DeepCopy() *BatchingStrategy

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BatchingStrategy.

func (*BatchingStrategy) DeepCopyInto

func (in *BatchingStrategy) DeepCopyInto(out *BatchingStrategy)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type FrameworkType

type FrameworkType string

const (
	FrameworkTFServing FrameworkType = "TFServing"
	FrameworkTriton    FrameworkType = "Triton"
)

type Inference

type Inference struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   InferenceSpec   `json:"spec,omitempty"`
	Status InferenceStatus `json:"status,omitempty"`
}

Inference is the Schema for the inference API.

func (*Inference) DeepCopy

func (in *Inference) DeepCopy() *Inference

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Inference.

func (*Inference) DeepCopyInto

func (in *Inference) DeepCopyInto(out *Inference)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*Inference) DeepCopyObject

func (in *Inference) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

type InferenceList

type InferenceList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []Inference `json:"items"`
}

InferenceList contains a list of Inference

func (*InferenceList) DeepCopy

func (in *InferenceList) DeepCopy() *InferenceList

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new InferenceList.

func (*InferenceList) DeepCopyInto

func (in *InferenceList) DeepCopyInto(out *InferenceList)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

func (*InferenceList) DeepCopyObject

func (in *InferenceList) DeepCopyObject() runtime.Object

DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.

type InferenceSpec

type InferenceSpec struct {
	// Framework specify the backend inference framework.
	Framework FrameworkType `json:"framework"`

	// Predictors defines a list of predictor specifications and serving-related strategies.
	Predictors []PredictorSpec `json:"predictors"`
}

InferenceSpec defines the desired state of Inference

func (*InferenceSpec) DeepCopy

func (in *InferenceSpec) DeepCopy() *InferenceSpec

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new InferenceSpec.

func (*InferenceSpec) DeepCopyInto

func (in *InferenceSpec) DeepCopyInto(out *InferenceSpec)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type InferenceStatus

type InferenceStatus struct {
	// InferenceEndpoints exposes available serving service endpoint.
	InferenceEndpoint string `json:"inferenceEndpoint,omitempty"`
	// PredictorStatuses exposes current observed status for each predictor.
	PredictorStatuses []PredictorStatus `json:"predictorStatuses,omitempty"`
}

InferenceStatus defines the observed state of Inference

func (*InferenceStatus) DeepCopy

func (in *InferenceStatus) DeepCopy() *InferenceStatus

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new InferenceStatus.

func (*InferenceStatus) DeepCopyInto

func (in *InferenceStatus) DeepCopyInto(out *InferenceStatus)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type PredictorSpec

type PredictorSpec struct {
	// Name indicates the predictor name.
	Name string `json:"name"`

	// ModelVersion specifies the name of target model-with-version to be loaded
	// in serving service, ModelVersion object has to be created before serving
	// service deployed.
	ModelVersion string `json:"modelVersion,omitempty"`

	// ModelPath is the loaded model filepath in container.
	ModelPath *string `json:"modelPath,omitempty"`

	// Replicas specify the expected predictor replicas.
	Replicas *int32 `json:"replicas,omitempty"`

	// TrafficWeight defines the traffic split weights across multiple predictors
	TrafficWeight *int32 `json:"trafficWeight,omitempty"`

	// Template describes a template of predictor pod with its properties.
	Template corev1.PodTemplateSpec `json:"template"`

	// AutoScale describes auto-scaling strategy for predictor.
	AutoScale *AutoScaleStrategy `json:"autoScale,omitempty"`

	// Batching specify batching strategy to control how to forward a batch of
	// aggregated inference requests to backend service.
	Batching *BatchingStrategy `json:"batching,omitempty"`
}

func (*PredictorSpec) DeepCopy

func (in *PredictorSpec) DeepCopy() *PredictorSpec

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PredictorSpec.

func (*PredictorSpec) DeepCopyInto

func (in *PredictorSpec) DeepCopyInto(out *PredictorSpec)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

type PredictorStatus

type PredictorStatus struct {
	// Name is the name of current predictor.
	Name string `json:"name"`
	// Replicas is the expected replicas of current predictor.
	Replicas int32 `json:"replicas"`
	// ReadyReplicas is the ready replicas of current predictor.
	ReadyReplicas int32 `json:"readyReplicas"`
	// TrafficPercent is the traffic distribution routed to this service.
	TrafficPercent *int32 `json:"trafficPercent,omitempty"`
	// Endpoint is the available endpoints of current predictor.
	Endpoint string `json:"endpoint"`
}

func (*PredictorStatus) DeepCopy

func (in *PredictorStatus) DeepCopy() *PredictorStatus

DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PredictorStatus.

func (*PredictorStatus) DeepCopyInto

func (in *PredictorStatus) DeepCopyInto(out *PredictorStatus)

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.