Documentation ¶
Index ¶
- Constants
- Variables
- func NewAssumeCache(informer cache.SharedIndexInformer, description, indexName string, ...) *assumeCache
- func RegisterVolumeSchedulingMetrics()
- type AssumeCache
- type ControllerParameters
- type FakeVolumeBinder
- func (b *FakeVolumeBinder) AssumePodVolumes(assumedPod *v1.Pod, nodeName string) (bool, error)
- func (b *FakeVolumeBinder) BindPodVolumes(assumedPod *v1.Pod) error
- func (b *FakeVolumeBinder) FindPodVolumes(pod *v1.Pod, node *v1.Node) (unboundVolumesSatisfied, boundVolumesSatsified bool, err error)
- func (b *FakeVolumeBinder) GetBindingsCache() PodBindingCache
- type FakeVolumeBinderConfig
- type PVAssumeCache
- type PVCAssumeCache
- type PersistentVolumeController
- func (ctrl *PersistentVolumeController) DeleteServiceAccountTokenFunc() func(types.UID)
- func (ctrl *PersistentVolumeController) GetCSIClient() csiclientset.Interface
- func (ctrl *PersistentVolumeController) GetCloudProvider() cloudprovider.Interface
- func (ctrl *PersistentVolumeController) GetConfigMapFunc() func(namespace, name string) (*v1.ConfigMap, error)
- func (ctrl *PersistentVolumeController) GetEventRecorder() record.EventRecorder
- func (adc *PersistentVolumeController) GetExec(pluginName string) mount.Exec
- func (ctrl *PersistentVolumeController) GetHostIP() (net.IP, error)
- func (ctrl *PersistentVolumeController) GetHostName() string
- func (ctrl *PersistentVolumeController) GetKubeClient() clientset.Interface
- func (ctrl *PersistentVolumeController) GetMounter(pluginName string) mount.Interface
- func (ctrl *PersistentVolumeController) GetNodeAllocatable() (v1.ResourceList, error)
- func (ctrl *PersistentVolumeController) GetNodeLabels() (map[string]string, error)
- func (ctrl *PersistentVolumeController) GetNodeName() types.NodeName
- func (ctrl *PersistentVolumeController) GetPluginDir(pluginName string) string
- func (ctrl *PersistentVolumeController) GetPodPluginDir(podUID types.UID, pluginName string) string
- func (ctrl *PersistentVolumeController) GetPodVolumeDeviceDir(ppodUID types.UID, pluginName string) string
- func (ctrl *PersistentVolumeController) GetPodVolumeDir(podUID types.UID, pluginName string, volumeName string) string
- func (ctrl *PersistentVolumeController) GetPodsDir() string
- func (ctrl *PersistentVolumeController) GetSecretFunc() func(namespace, name string) (*v1.Secret, error)
- func (ctrl *PersistentVolumeController) GetServiceAccountTokenFunc() ...
- func (ctrl *PersistentVolumeController) GetVolumeDevicePluginDir(pluginName string) string
- func (ctrl *PersistentVolumeController) NewWrapperMounter(volName string, spec vol.Spec, pod *v1.Pod, opts vol.VolumeOptions) (vol.Mounter, error)
- func (ctrl *PersistentVolumeController) NewWrapperUnmounter(volName string, spec vol.Spec, podUID types.UID) (vol.Unmounter, error)
- func (ctrl *PersistentVolumeController) Run(stopCh <-chan struct{})
- type PodBindingCache
- type SchedulerVolumeBinder
Constants ¶
const CloudVolumeCreatedForClaimNameTag = "kubernetes.io/created-for/pvc/name"
CloudVolumeCreatedForClaimNameTag is a name of a tag attached to a real volume in cloud (e.g. AWS EBS or GCE PD) with name of a persistent volume claim used to create this volume.
const CloudVolumeCreatedForClaimNamespaceTag = "kubernetes.io/created-for/pvc/namespace"
CloudVolumeCreatedForClaimNamespaceTag is a name of a tag attached to a real volume in cloud (e.g. AWS EBS or GCE PD) with namespace of a persistent volume claim used to create this volume.
const CloudVolumeCreatedForVolumeNameTag = "kubernetes.io/created-for/pv/name"
CloudVolumeCreatedForVolumeNameTag is a name of a tag attached to a real volume in cloud (e.g. AWS EBS or GCE PD) with name of appropriate Kubernetes persistent volume .
const VolumeSchedulerSubsystem = "scheduler_volume"
VolumeSchedulerSubsystem - subsystem name used by scheduler
Variables ¶
var ( VolumeBindingRequestSchedulerBinderCache = prometheus.NewCounterVec( prometheus.CounterOpts{ Subsystem: VolumeSchedulerSubsystem, Name: "binder_cache_requests_total", Help: "Total number for request volume binding cache", }, []string{"operation"}, ) VolumeSchedulingStageLatency = prometheus.NewHistogramVec( prometheus.HistogramOpts{ Subsystem: VolumeSchedulerSubsystem, Name: "scheduling_duration_seconds", Help: "Volume scheduling stage latency", Buckets: prometheus.ExponentialBuckets(1000, 2, 15), }, []string{"operation"}, ) VolumeSchedulingStageFailed = prometheus.NewCounterVec( prometheus.CounterOpts{ Subsystem: VolumeSchedulerSubsystem, Name: "scheduling_stage_error_total", Help: "Volume scheduling stage error count", }, []string{"operation"}, ) )
Functions ¶
func NewAssumeCache ¶ added in v1.9.0
func NewAssumeCache(informer cache.SharedIndexInformer, description, indexName string, indexFunc cache.IndexFunc) *assumeCache
func RegisterVolumeSchedulingMetrics ¶ added in v1.13.1
func RegisterVolumeSchedulingMetrics()
RegisterVolumeSchedulingMetrics is used for scheduler, because the volume binding cache is a library used by scheduler process.
Types ¶
type AssumeCache ¶ added in v1.9.0
type AssumeCache interface { // Assume updates the object in-memory only Assume(obj interface{}) error // Restore the informer cache's version of the object Restore(objName string) // Get the object by name Get(objName string) (interface{}, error) // List all the objects in the cache List(indexObj interface{}) []interface{} }
AssumeCache is a cache on top of the informer that allows for updating objects outside of informer events and also restoring the informer cache's version of the object. Objects are assumed to be Kubernetes API objects that implement meta.Interface
type ControllerParameters ¶ added in v1.5.1
type ControllerParameters struct { KubeClient clientset.Interface SyncPeriod time.Duration VolumePlugins []vol.VolumePlugin Cloud cloudprovider.Interface ClusterName string VolumeInformer coreinformers.PersistentVolumeInformer ClaimInformer coreinformers.PersistentVolumeClaimInformer ClassInformer storageinformers.StorageClassInformer PodInformer coreinformers.PodInformer NodeInformer coreinformers.NodeInformer EventRecorder record.EventRecorder EnableDynamicProvisioning bool }
ControllerParameters contains arguments for creation of a new PersistentVolume controller.
type FakeVolumeBinder ¶ added in v1.9.0
type FakeVolumeBinder struct { AssumeCalled bool BindCalled bool // contains filtered or unexported fields }
func NewFakeVolumeBinder ¶ added in v1.9.0
func NewFakeVolumeBinder(config *FakeVolumeBinderConfig) *FakeVolumeBinder
NewVolumeBinder sets up all the caches needed for the scheduler to make topology-aware volume binding decisions.
func (*FakeVolumeBinder) AssumePodVolumes ¶ added in v1.9.0
func (*FakeVolumeBinder) BindPodVolumes ¶ added in v1.9.0
func (b *FakeVolumeBinder) BindPodVolumes(assumedPod *v1.Pod) error
func (*FakeVolumeBinder) FindPodVolumes ¶ added in v1.9.0
func (*FakeVolumeBinder) GetBindingsCache ¶ added in v1.9.0
func (b *FakeVolumeBinder) GetBindingsCache() PodBindingCache
type FakeVolumeBinderConfig ¶ added in v1.9.0
type PVAssumeCache ¶ added in v1.9.0
type PVAssumeCache interface { AssumeCache GetPV(pvName string) (*v1.PersistentVolume, error) ListPVs(storageClassName string) []*v1.PersistentVolume }
PVAssumeCache is a AssumeCache for PersistentVolume objects
func NewPVAssumeCache ¶ added in v1.9.0
func NewPVAssumeCache(informer cache.SharedIndexInformer) PVAssumeCache
type PVCAssumeCache ¶ added in v1.11.2
type PVCAssumeCache interface { AssumeCache // GetPVC returns the PVC from the cache with the same // namespace and the same name of the specified pod. // pvcKey is the result of MetaNamespaceKeyFunc on PVC obj GetPVC(pvcKey string) (*v1.PersistentVolumeClaim, error) }
PVCAssumeCache is a AssumeCache for PersistentVolumeClaim objects
func NewPVCAssumeCache ¶ added in v1.11.2
func NewPVCAssumeCache(informer cache.SharedIndexInformer) PVCAssumeCache
type PersistentVolumeController ¶
type PersistentVolumeController struct { NodeLister corelisters.NodeLister NodeListerSynced cache.InformerSynced // contains filtered or unexported fields }
PersistentVolumeController is a controller that synchronizes PersistentVolumeClaims and PersistentVolumes. It starts two cache.Controllers that watch PersistentVolume and PersistentVolumeClaim changes.
func NewController ¶ added in v1.5.1
func NewController(p ControllerParameters) (*PersistentVolumeController, error)
NewController creates a new PersistentVolume controller
func (*PersistentVolumeController) DeleteServiceAccountTokenFunc ¶ added in v1.13.1
func (ctrl *PersistentVolumeController) DeleteServiceAccountTokenFunc() func(types.UID)
func (*PersistentVolumeController) GetCSIClient ¶ added in v1.12.0
func (ctrl *PersistentVolumeController) GetCSIClient() csiclientset.Interface
func (*PersistentVolumeController) GetCloudProvider ¶
func (ctrl *PersistentVolumeController) GetCloudProvider() cloudprovider.Interface
func (*PersistentVolumeController) GetConfigMapFunc ¶ added in v1.7.0
func (ctrl *PersistentVolumeController) GetConfigMapFunc() func(namespace, name string) (*v1.ConfigMap, error)
func (*PersistentVolumeController) GetEventRecorder ¶ added in v1.11.2
func (ctrl *PersistentVolumeController) GetEventRecorder() record.EventRecorder
func (*PersistentVolumeController) GetExec ¶ added in v1.8.0
func (adc *PersistentVolumeController) GetExec(pluginName string) mount.Exec
func (*PersistentVolumeController) GetHostIP ¶
func (ctrl *PersistentVolumeController) GetHostIP() (net.IP, error)
func (*PersistentVolumeController) GetHostName ¶
func (ctrl *PersistentVolumeController) GetHostName() string
func (*PersistentVolumeController) GetKubeClient ¶
func (ctrl *PersistentVolumeController) GetKubeClient() clientset.Interface
func (*PersistentVolumeController) GetMounter ¶
func (ctrl *PersistentVolumeController) GetMounter(pluginName string) mount.Interface
func (*PersistentVolumeController) GetNodeAllocatable ¶ added in v1.4.2
func (ctrl *PersistentVolumeController) GetNodeAllocatable() (v1.ResourceList, error)
func (*PersistentVolumeController) GetNodeLabels ¶ added in v1.7.0
func (ctrl *PersistentVolumeController) GetNodeLabels() (map[string]string, error)
func (*PersistentVolumeController) GetNodeName ¶ added in v1.9.0
func (ctrl *PersistentVolumeController) GetNodeName() types.NodeName
func (*PersistentVolumeController) GetPluginDir ¶
func (ctrl *PersistentVolumeController) GetPluginDir(pluginName string) string
func (*PersistentVolumeController) GetPodPluginDir ¶
func (ctrl *PersistentVolumeController) GetPodPluginDir(podUID types.UID, pluginName string) string
func (*PersistentVolumeController) GetPodVolumeDeviceDir ¶ added in v1.9.0
func (ctrl *PersistentVolumeController) GetPodVolumeDeviceDir(ppodUID types.UID, pluginName string) string
func (*PersistentVolumeController) GetPodVolumeDir ¶
func (*PersistentVolumeController) GetPodsDir ¶ added in v1.11.2
func (ctrl *PersistentVolumeController) GetPodsDir() string
func (*PersistentVolumeController) GetSecretFunc ¶ added in v1.6.0
func (ctrl *PersistentVolumeController) GetSecretFunc() func(namespace, name string) (*v1.Secret, error)
func (*PersistentVolumeController) GetServiceAccountTokenFunc ¶ added in v1.11.2
func (ctrl *PersistentVolumeController) GetServiceAccountTokenFunc() func(_, _ string, _ *authenticationv1.TokenRequest) (*authenticationv1.TokenRequest, error)
func (*PersistentVolumeController) GetVolumeDevicePluginDir ¶ added in v1.9.0
func (ctrl *PersistentVolumeController) GetVolumeDevicePluginDir(pluginName string) string
func (*PersistentVolumeController) NewWrapperMounter ¶
func (*PersistentVolumeController) NewWrapperUnmounter ¶
func (*PersistentVolumeController) Run ¶
func (ctrl *PersistentVolumeController) Run(stopCh <-chan struct{})
Run starts all of this controller's control loops
type PodBindingCache ¶ added in v1.9.0
type PodBindingCache interface { // UpdateBindings will update the cache with the given bindings for the // pod and node. UpdateBindings(pod *v1.Pod, node string, bindings []*bindingInfo) // GetBindings will return the cached bindings for the given pod and node. // A nil return value means that the entry was not found. An empty slice // means that no binding operations are needed. GetBindings(pod *v1.Pod, node string) []*bindingInfo // UpdateProvisionedPVCs will update the cache with the given provisioning decisions // for the pod and node. UpdateProvisionedPVCs(pod *v1.Pod, node string, provisionings []*v1.PersistentVolumeClaim) // GetProvisionedPVCs will return the cached provisioning decisions for the given pod and node. // A nil return value means that the entry was not found. An empty slice // means that no provisioning operations are needed. GetProvisionedPVCs(pod *v1.Pod, node string) []*v1.PersistentVolumeClaim // DeleteBindings will remove all cached bindings and provisionings for the given pod. // TODO: separate the func if it is needed to delete bindings/provisionings individually DeleteBindings(pod *v1.Pod) }
podBindingCache stores PV binding decisions per pod per node. Pod entries are removed when the Pod is deleted or updated to no longer be schedulable.
func NewPodBindingCache ¶ added in v1.9.0
func NewPodBindingCache() PodBindingCache
type SchedulerVolumeBinder ¶ added in v1.9.0
type SchedulerVolumeBinder interface { // FindPodVolumes checks if all of a Pod's PVCs can be satisfied by the node. // // If a PVC is bound, it checks if the PV's NodeAffinity matches the Node. // Otherwise, it tries to find an available PV to bind to the PVC. // // It returns true if all of the Pod's PVCs have matching PVs or can be dynamic provisioned, // and returns true if bound volumes satisfy the PV NodeAffinity. // // This function is called by the volume binding scheduler predicate and can be called in parallel FindPodVolumes(pod *v1.Pod, node *v1.Node) (unboundVolumesSatisified, boundVolumesSatisfied bool, err error) // AssumePodVolumes will: // 1. Take the PV matches for unbound PVCs and update the PV cache assuming // that the PV is prebound to the PVC. // 2. Take the PVCs that need provisioning and update the PVC cache with related // annotations set. // // It returns true if all volumes are fully bound // // This function will modify assumedPod with the node name. // This function is called serially. AssumePodVolumes(assumedPod *v1.Pod, nodeName string) (allFullyBound bool, err error) // BindPodVolumes will: // 1. Initiate the volume binding by making the API call to prebind the PV // to its matching PVC. // 2. Trigger the volume provisioning by making the API call to set related // annotations on the PVC // 3. Wait for PVCs to be completely bound by the PV controller // // This function can be called in parallel. BindPodVolumes(assumedPod *v1.Pod) error // GetBindingsCache returns the cache used (if any) to store volume binding decisions. GetBindingsCache() PodBindingCache }
SchedulerVolumeBinder is used by the scheduler to handle PVC/PV binding and dynamic provisioning. The binding decisions are integrated into the pod scheduling workflow so that the PV NodeAffinity is also considered along with the pod's other scheduling requirements.
This integrates into the existing default scheduler workflow as follows:
- The scheduler takes a Pod off the scheduler queue and processes it serially: a. Invokes all predicate functions, parallelized across nodes. FindPodVolumes() is invoked here. b. Invokes all priority functions. Future/TBD c. Selects the best node for the Pod. d. Cache the node selection for the Pod. AssumePodVolumes() is invoked here. i. If PVC binding is required, cache in-memory only: * For manual binding: update PV objects for prebinding to the corresponding PVCs. * For dynamic provisioning: update PVC object with a selected node from c) * For the pod, which PVCs and PVs need API updates. ii. Afterwards, the main scheduler caches the Pod->Node binding in the scheduler's pod cache, This is handled in the scheduler and not here. e. Asynchronously bind volumes and pod in a separate goroutine i. BindPodVolumes() is called first. It makes all the necessary API updates and waits for PV controller to fully bind and provision the PVCs. If binding fails, the Pod is sent back through the scheduler. ii. After BindPodVolumes() is complete, then the scheduler does the final Pod->Node binding.
- Once all the assume operations are done in d), the scheduler processes the next Pod in the scheduler queue while the actual binding operation occurs in the background.
func NewVolumeBinder ¶ added in v1.9.0
func NewVolumeBinder( kubeClient clientset.Interface, pvcInformer coreinformers.PersistentVolumeClaimInformer, pvInformer coreinformers.PersistentVolumeInformer, storageClassInformer storageinformers.StorageClassInformer, bindTimeout time.Duration) SchedulerVolumeBinder
NewVolumeBinder sets up all the caches needed for the scheduler to make volume binding decisions.