dra

README

Overview

The tests in this directory cover dynamic resource allocation support in Kubernetes. They do not test the correct behavior of arbitrary dynamic resource allocation drivers.

If such a driver is needed, then the in-tree test/e2e/dra/test-driver is used, with a slight twist: instead of deploying that driver directly in the cluster, the necessary sockets for interaction with kubelet (registration and dynamic resource allocation) get proxied into the e2e.test binary. This reuses the work done for CSI mock testing. The advantage is that no separate images are needed for the test driver and that the e2e test has full control over all gRPC calls, in case that it needs that for operations like error injection or checking calls.

Cluster setup preparation

The container runtime must support CDI. CRI-O supports CDI starting from release 1.23, Containerd supports CDI starting from release 1.7. To bring up a Kind cluster with Containerd, two things are needed:

• build binaries from Kubernetes source code tree
• Kind

NB: Kind switched to use worker-node base image with Containerd 1.7 by default starting from release 0.20, build kind from latest main branch sources or use Kind release binary 0.20 or later.

Build kind node image

After building Kubernetes, in Kubernetes source code tree build new node image:

$ kind build node-image --image dra/node:latest $(pwd)

Bring up a Kind cluster

$ kind create cluster --config test/e2e/dra/kind.yaml --image dra/node:latest

Run tests

• Build ginkgo

$ make ginkgo

• Run e2e tests for the Dynamic Resource Allocation feature:

$ KUBECONFIG=~/.kube/config _output/bin/ginkgo -p -v -focus=Feature:DynamicResourceAllocation ./test/e2e

Documentation

Index

• Constants
• type Driver
  • func NewDriver(f *framework.Framework, nodes *Nodes, configureResources func() Resources, ...) *Driver
  • func NewDriverInstance(f *framework.Framework) *Driver
  • func (d *Driver) CallCount(m MethodInstance) int64
  • func (d *Driver) Fail(m MethodInstance, injectError bool)
  • func (d *Driver) IsGone(ctx context.Context)
  • func (d *Driver) Nodenames() (nodenames []string)
  • func (d *Driver) Run(nodes *Nodes, configureResources func() Resources, ...)
  • func (d *Driver) SetUp(nodes *Nodes, resources Resources, ...)
  • func (d *Driver) TearDown()
• type KubeletPlugin
• type MethodInstance
• type Nodes
  • func NewNodes(f *framework.Framework, minNodes, maxNodes int) *Nodes
  • func NewNodesNow(ctx context.Context, f *framework.Framework, minNodes, maxNodes int) *Nodes
• type Resources

Constants

const (
	NodePrepareResourcesMethod   = "/k8s.io.kubelet.pkg.apis.dra.v1beta1.DRAPlugin/NodePrepareResources"
	NodeUnprepareResourcesMethod = "/k8s.io.kubelet.pkg.apis.dra.v1beta1.DRAPlugin/NodeUnprepareResources"
)

Types

type Driver

type Driver struct {
	NameSuffix string
	Name       string

	// Nodes contains entries for each node selected for a test when the test runs.
	// In addition, there is one entry for a fictional node.
	Nodes map[string]KubeletPlugin

	NodeV1alpha4 bool
	NodeV1beta1  bool
	// contains filtered or unexported fields
}

func NewDriver

func NewDriver(f *framework.Framework, nodes *Nodes, configureResources func() Resources, devicesPerNode ...map[string]map[resourceapi.QualifiedName]resourceapi.DeviceAttribute) *Driver

NewDriver sets up controller (as client of the cluster) and kubelet plugin (via proxy) before the test runs. It cleans up after the test.

Call this outside of ginkgo.It, then use the instance inside ginkgo.It.

func NewDriverInstance

func NewDriverInstance(f *framework.Framework) *Driver

NewDriverInstance is a variant of NewDriver where the driver is inactive and must be started explicitly with Run. May be used inside ginkgo.It.

func (*Driver) CallCount

func (d *Driver) CallCount(m MethodInstance) int64

func (*Driver) Fail

func (d *Driver) Fail(m MethodInstance, injectError bool)

func (*Driver) IsGone

func (d *Driver) IsGone(ctx context.Context)

func (*Driver) Nodenames

func (d *Driver) Nodenames() (nodenames []string)

func (*Driver) Run

func (d *Driver) Run(nodes *Nodes, configureResources func() Resources, devicesPerNode ...map[string]map[resourceapi.QualifiedName]resourceapi.DeviceAttribute)

func (*Driver) SetUp

func (d *Driver) SetUp(nodes *Nodes, resources Resources, devicesPerNode ...map[string]map[resourceapi.QualifiedName]resourceapi.DeviceAttribute)

func (*Driver) TearDown

func (d *Driver) TearDown()

type KubeletPlugin

type KubeletPlugin struct {
	*app.ExamplePlugin
	ClientSet kubernetes.Interface
}

type MethodInstance

type MethodInstance struct {
	Nodename   string
	FullMethod string
}

type Nodes

type Nodes struct {
	NodeNames []string
}

func NewNodes

func NewNodes(f *framework.Framework, minNodes, maxNodes int) *Nodes

NewNodes selects nodes to run the test on.

Call this outside of ginkgo.It, then use the instance inside ginkgo.It.

func NewNodesNow

func NewNodesNow(ctx context.Context, f *framework.Framework, minNodes, maxNodes int) *Nodes

NewNodesNow is a variant of NewNodes which can be used inside a ginkgo.It.

type Resources

type Resources struct {
	NodeLocal bool

	// Nodes is a fixed list of node names on which resources are
	// available. Mutually exclusive with NodeLabels.
	Nodes []string

	// Number of devices called "device-000", "device-001", ... on each node or in the cluster.
	MaxAllocations int
}