kutest

README

Kutest

Kutest is a Ginkgo helper library to help you write tests for your Kubernetes clusters.

It aims to be easy to operate in the following ways:

• Nothing is required to be deployed on the cluster to run tests.
• Readable test code that cuts down on the noise and gets to the point of the test.
• You can run the suite from your local machine or from the cluster depending on your use-case. It works the same. This brings you a nice local development loop to iterate on changes without the logic being too different in a continuous integration context.
• Parallelism-friendly to speed up test suite performance.

Here's how it looks:

Specify("hello world", func() {
    WithJob(JobOptions{Namespace: "default"}, func() {
        fmt.Fprintln(GinkgoWriter, "hello from pod!")
    })
})

Read kutest_test.go for examples, such as testing connectivity between two pods.

Usage

Install to your project: go get github.com/lanvstn/kutest

You'll want to package your test as a container image. Read Dockerfile for an example.

Here's a quick example with Kind:

> go install github.com/onsi/ginkgo/v2/ginkgo

> export KUTEST_IMAGE="localhost/kutest-example:latest"
> export KUTEST_DEFAULTIMAGEPULLPOLICY="IfNotPresent" # Only for Kind clusters.

> docker build . -t "$KUTEST_IMAGE" && \
    kind load docker-image "$KUTEST_IMAGE" && \
    ginkgo run --json-report=report.json -v

> go run ./cmd/kutesthtml < report.json > report.html

Note on image settings

It's very important that whatever is running locally (on the controller) is the same as on the pods spawned by Kutest due to how Kutest identifies its pods (using runtime.Callers). Keep this in mind when building images.

• If KUTEST_IMAGE is a tagged with something you're planning to overwrite, don't set KUTEST_DEFAULTIMAGEPULLPOLICY.
• Make sure others are not pushing the same KUTEST_IMAGE during your testing.

Documentation

Pitfalls

This is a weekend project. I can fix some of these up. But for now these are easier to document. Watch out for:

• Don't WithJob from inside a WithJob!
  • I can prevent this from being possible but have not added this safeguard
  • A WithJob should be able to WithJob in the future. One scenario is testing the running of jobs from a job (RBAC). However, currently this is not possible due to the single-controller design.

Using with different test frameworks

You may have an exisiting test suite. For example, let's say you're doing continuous load testing using K6.

This is do-able with Kutest as well: you don't have to implement everything in Go.

1. Write a Go helper around your test tool
2. Ensure the dependencies are packaged in your test container image
3. Use your helper. E.g.:

It("passes the K6 loadtest", func() {
    WithJob(JobOptions{
        Namespace: "default",
        // TODO: may want to override some resource limits here
    }, func() {
        // TODO: implement K6Helper, which runs a local k6 binary with the given file
        K6Helper("my_loadtest.js")
    })
})

About reporting

Ginkgo has a built-in report generation system. You can use its output by passing --json-report=report.json to gingko run.

./cmd/kutesthtml provides a basic html/template coverter for this JSON format that uses stdin and stdout.

On the resulting page you can view the result of the suite, every test, and the logs of the jobs that were started by Kutest.

WithJob

WithJob makes your test suite replicate itself throughout your cluster. Based on the current pod name, the library is aware whether it is running inside the job where the passed function is desired to run.

The first instance of the running test suite is called the controller. This is determined by the KUTEST_SESSID environment variable: if it is set, the current instance is part of an existing session and cannot be the controller. The controller creates the session ID and adds it through the environment variable to each job it creates.

This gives the appearance of the test function being magically sent to a new job for execution, but without actually sending the code around and all complexity that comes with it.

It is not recommended to pass anything out of your WithJob call since the function you pass to it will not always run, but the other logic in your test will!

Project status

• Weekend project.
• Experimental library with unstable API.
• Not known to be used in production.
• Accepting contributions but make an issue before starting a PR.

---

Made with :3 by Lander

Documentation

Index

• func KutestSetup() error
• func WithJob(opts JobOptions, f func())
• type ConfigSpec
• type ExposeOptions
• type JobOptions

Functions

func KutestSetup

func KutestSetup() error

func WithJob

func WithJob(opts JobOptions, f func())

WithJob runs f inside a new pod specified by PodOptions. If the pod fails or anything goes wrong it will call Fail on Ginkgo. Please be aware that since f executes on another pod, you are not able to pass values outside of f!

- When on a controller, this runs the job to completion.

- When on the job, this runs f.

- When on another job, this does nothing.

Types

type ConfigSpec

type ConfigSpec struct {
	// The path to the kubeconfig to use
	KubeconfigPath string

	// The image name which will be used by the tests to schedule copies of themselves to jobs
	Image string `required:"true"`

	// The UID of the user in the scheduled jobs
	UID int64 `default:"1000"`

	// The image pull policy set in the scheduled jobs.
	// Use `IfNotPresent` when using `kind load` and local images.
	DefaultImagePullPolicy string `default:"Always"`

	// SessID is the sessionID passed from the controller.
	// You should not set this yourself.
	SessID string

	// PodName is the name of this pod using Kubernetes envvar projection.
	// You should not set this yourself.
	PodName string
}

var (
	// Config is populated after KutestSetup
	Config ConfigSpec
)

type ExposeOptions

type ExposeOptions struct {
	// Name is REQUIRED and will recieve the name of the service (when created if on controller)
	//
	// TODO:
	// 	This feels slightly awkward but is due the fact that the name creation is a Kutest internal detail.
	// 	Maybe this should be reconsidered to create a more natural interface.
	Name chan<- string

	// Port is both the service and the target port.
	Port int32
}

type JobOptions

type JobOptions struct {
	Namespace      string
	ServiceAccount string
	Labels         map[string]string

	// MutateJob apply transformations to the pod that would be created
	MutateJob func(*batchv1.Job) *batchv1.Job

	// Expose, if set, will create a service for the job.
	Expose *ExposeOptions
}