nodelabel

README

Node Label Sync Controller

The Node Label Sync Controller is responsible for syncing labels set on Kubernetes nodes to the StorageOS node objects.

Some StorageOS functionality, such as marking a node as "compute only", is done by setting the storageos.com/computeonly=true label on a Kubernetes node. This controller ensures that the behaviour is applied to the StorageOS node.

Trigger

The controller reconcile will trigger on any Kubernetes Node label update event where the node has the StorageOS CSI driver annotation.

The annotation is added by the CSI node driver registrar when StorageOS starts on the node. Once added, it is not removed.

The controller reconcile will also be triggered when the StorageOS CSI driver annotation is added to a node. This allows the existing node labels to be applied almost immediately to a node when it starts running StorageOS for the first time.

Reconcile

When the labels on a Kubernetes node with the StorageOS CSI driver annotation is updated, a request is made to the StorageOS API to re-apply the labels to the StorageOS node.

Labels prefixed with storageos.com/ have special meaning, and will likely be applied with a discrete call to the StorageOS API. This ensures that the behaviour can be applied in a strongly-consistent manner or return an error.

Remaining labels without the storageos.com/ prefix will be applied as a single API call. They have no internal meaning to StorageOS but they can be used to influence placement decisions.

If a nodel label sync fails, it will be requeued and retried after a backoff period. It is possible that the application of only a partial set of labels will succeed. If StorageOS can't apply a certain behaviour change (for example, if the change would result in a volume going offline), then only that behaviour change would fail and the remaining changes would be attempted. If any change fails, the whole set of labels will be retried until they all succeed.

Resync

In case a node label update event was missed during a restart or outage, a resync runs periodically. It re-applies the set of Kubernetes node labels to StorageOS nodes.

Node label resync is run every hour by default (configurable via the -node-label-resync-interval flag). It can be disabled by setting -node-label-resync-interval to 0s.

Resync is run on startup after a delay defined by the -node-label-resync-delay flag.

Disabling

The Node Label Sync Controller can be disabled by setting the -enable-node-label-sync=false flag.

Documentation

Index

• type Controller
  • func NewController(api NodeLabeller, log logr.Logger) (*Controller, error)
  • func (c Controller) Delete(ctx context.Context, obj client.Object) error
  • func (c Controller) Diff(ctx context.Context, objs []client.Object) ([]client.Object, error)
  • func (c Controller) Ensure(ctx context.Context, obj client.Object) error
• type NodeLabeller
• type Predicate
  • func (p Predicate) Update(e event.UpdateEvent) bool
• type Reconciler
  • func NewReconciler(api NodeLabeller, k8s client.Client, resyncDelay time.Duration, ...) *Reconciler
  • func (r *Reconciler) SetupWithManager(mgr ctrl.Manager, workers int) error

Types

type Controller

type Controller struct {
	// contains filtered or unexported fields
}

Controller implements the Sync contoller interface, applying node labels to StorageOS nodes.

func NewController

func NewController(api NodeLabeller, log logr.Logger) (*Controller, error)

NewController returns a Controller that implements node label sync in StorageOS.

func (Controller) Delete

func (c Controller) Delete(ctx context.Context, obj client.Object) error

Delete is a no-op. The node-delete controller will handle deletes.

func (Controller) Diff

func (c Controller) Diff(ctx context.Context, objs []client.Object) ([]client.Object, error)

Diff takes a list of Kubernets node objects and returns them if they exist within StorageOS but the labels are different.

func (Controller) Ensure

func (c Controller) Ensure(ctx context.Context, obj client.Object) error

Ensure applies labels set on the k8s node to the StorageOS node.

StorageOS reserved labels are validated and applied first, then the remaining unreserved labels are applied.

Any errors will result in a requeue, with standard back-off retries.

There is no label sync from StorageOS to Kubernetes. This is intentional to ensure a simple flow of desired state set by users in Kubernetes to actual state set on the StorageOS node.

type NodeLabeller

type NodeLabeller interface {
	EnsureNodeLabels(ctx context.Context, key client.ObjectKey, labels map[string]string) error
	NodeObjects(ctx context.Context) (map[client.ObjectKey]storageos.Object, error)
}

NodeLabeller provides access to update node labels.

type Predicate

type Predicate struct {
	predicate.IgnoreFuncs
	// contains filtered or unexported fields
}

Predicate filters events before enqueuing the keys. Ignore all but Update events, and then filter out events from non-StorageOS nodes. Trigger a resync when labels have changed.

Nodes added to the cluster will not immediately be added to StorageOS, so we can't react to node create events. Instead, trigger a resync when the StorageOS CSI driver annotation has been added, indicating that the node is known to StorageOS and can receive label updates.

func (Predicate) Update

func (p Predicate) Update(e event.UpdateEvent) bool

Update determines whether an object update should trigger a reconcile.

type Reconciler

type Reconciler struct {
	client.Client

	msyncv1.Reconciler
	// contains filtered or unexported fields
}

Reconciler reconciles a Node object by applying labels from the Kubernetes node to the StorageOS node object.

func NewReconciler

func NewReconciler(api NodeLabeller, k8s client.Client, resyncDelay time.Duration, resyncInterval time.Duration) *Reconciler

NewReconciler returns a new Node label reconciler.

The resyncInterval determines how often the periodic resync operation should be run.

func (*Reconciler) SetupWithManager

func (r *Reconciler) SetupWithManager(mgr ctrl.Manager, workers int) error

SetupWithManager registers the controller with the controller manager.