README
¶
KServe
KServe provides a Kubernetes Custom Resource Definition for serving machine learning (ML) models on arbitrary frameworks. It aims to solve production model serving use cases by providing performant, high abstraction interfaces for common ML frameworks like Tensorflow, XGBoost, ScikitLearn, PyTorch, and ONNX.
It encapsulates the complexity of autoscaling, networking, health checking, and server configuration to bring cutting edge serving features like GPU Autoscaling, Scale to Zero, and Canary Rollouts to your ML deployments. It enables a simple, pluggable, and complete story for Production ML Serving including prediction, pre-processing, post-processing and explainability. KServe is being used across various organizations.
For more details, visit KServe website
Since 0.7 KFServing is rebranded to KServe, we still support previous KFServing 0.5.x and 0.6.x releases, please refer to corresponding release branch for docs.
Learn More
To learn more about KServe, how to deploy it as part of Kubeflow, how to use various supported features, and how to participate in the KServe community, please follow the KServe website documentation. Additionally, we have compiled a list of presentations and demoes to dive through various details.
Installation
Standalone Installation
KServe by default installs Knative for serverless deployment, please follow Serverless installation guide to install KServe. If you are looking to install KServe without Knative(this feature is still alpha), please follow Raw Kubernetes Deployment installation guide.
Quick Install
Please follow quick install to install KServe on your local machine.
Create test inference service
Please follow getting started to create your first InferenceService.
Roadmap
API Reference
InferenceService v1beta1 API Docs
Developer Guide
Contributor Guide
Adopters
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
pkg
|
|
|
apis
Package apis contains Kubernetes API groups.
|
Package apis contains Kubernetes API groups. |
|
apis/serving/v1alpha1
Package v1alpha1 contains API Schema definitions for the serving v1alpha1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=kserve/pkg/apis/serving +k8s:defaulter-gen=TypeMeta +groupName=serving.kserve.io
|
Package v1alpha1 contains API Schema definitions for the serving v1alpha1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=kserve/pkg/apis/serving +k8s:defaulter-gen=TypeMeta +groupName=serving.kserve.io |
|
apis/serving/v1beta1
Package v1beta1 contains API Schema definitions for the serving v1beta1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=kserve/pkg/apis/serving +k8s:defaulter-gen=TypeMeta +groupName=serving.kserve.io
|
Package v1beta1 contains API Schema definitions for the serving v1beta1 API group +k8s:openapi-gen=true +k8s:deepcopy-gen=package,register +k8s:conversion-gen=kserve/pkg/apis/serving +k8s:defaulter-gen=TypeMeta +groupName=serving.kserve.io |
|
client/clientset/versioned
This package has the automatically generated clientset.
|
This package has the automatically generated clientset. |
|
client/clientset/versioned/fake
This package has the automatically generated fake clientset.
|
This package has the automatically generated fake clientset. |
|
client/clientset/versioned/scheme
This package contains the scheme of the automatically generated clientset.
|
This package contains the scheme of the automatically generated clientset. |
|
client/clientset/versioned/typed/serving/v1beta1
This package has the automatically generated typed clients.
|
This package has the automatically generated typed clients. |
|
client/clientset/versioned/typed/serving/v1beta1/fake
Package fake has the automatically generated clients.
|
Package fake has the automatically generated clients. |
|
controller/v1alpha1/inferencegraph
+kubebuilder:rbac:groups=serving.kserve.io,resources=inferencegraphs,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=serving.kserve.io,resources=inferencegraphs/status,verbs=get;update;patch +kubebuilder:rbac:groups=serving.knative.dev,resources=services,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=serving.knative.dev,resources=services/finalizers,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=serving.knative.dev,resources=services/status,verbs=get;update;patch
|
+kubebuilder:rbac:groups=serving.kserve.io,resources=inferencegraphs,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=serving.kserve.io,resources=inferencegraphs/status,verbs=get;update;patch +kubebuilder:rbac:groups=serving.knative.dev,resources=services,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=serving.knative.dev,resources=services/finalizers,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=serving.knative.dev,resources=services/status,verbs=get;update;patch |
|
controller/v1alpha1/trainedmodel
+kubebuilder:rbac:groups=serving.kserve.io,resources=trainedmodels,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=serving.kserve.io,resources=trainedmodels/status,verbs=get;update;patch +kubebuilder:rbac:groups=serving.knative.dev,resources=services,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=serving.knative.dev,resources=services/status,verbs=get;update;patch +kubebuilder:rbac:groups=core,resources=serviceaccounts,verbs=get;list;watch +kubebuilder:rbac:groups=core,resources=configmaps,verbs=get;list;watch;update +kubebuilder:rbac:groups=core,resources=secrets,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=core,resources=namespaces,verbs=get;list;watch +kubebuilder:rbac:groups=core,resources=events,verbs=get;list;watch;create;update;patch;delete
|
+kubebuilder:rbac:groups=serving.kserve.io,resources=trainedmodels,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=serving.kserve.io,resources=trainedmodels/status,verbs=get;update;patch +kubebuilder:rbac:groups=serving.knative.dev,resources=services,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=serving.knative.dev,resources=services/status,verbs=get;update;patch +kubebuilder:rbac:groups=core,resources=serviceaccounts,verbs=get;list;watch +kubebuilder:rbac:groups=core,resources=configmaps,verbs=get;list;watch;update +kubebuilder:rbac:groups=core,resources=secrets,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=core,resources=namespaces,verbs=get;list;watch +kubebuilder:rbac:groups=core,resources=events,verbs=get;list;watch;create;update;patch;delete |
|
qpext
module
|
|
|
tools
|
|