ktesting

Documentation

Overview

Package ktesting is a wrapper around k8s.io/klog/v2/ktesting. In contrast to the klog package, this one is opinionated and tailored towards testing Kubernetes.

Importing it - adds the -v command line flag - enables better dumping of complex datatypes - sets the default verbosity to 5 (can be changed with SetDefaultVerbosity)

It also adds additional APIs and types for unit and integration tests which are too experimental for klog and/or are unrelated to logging. The ktesting package itself takes care of managing a test context with deadlines, timeouts, cancellation, and some common attributes as first-class members of the API. Sub-packages have additional APIs for propagating values via the context, implemented via WithValue.

Index

• Constants
• Variables
• func Consistently[T any](tCtx TContext, cb func(TContext) T) gomega.AsyncAssertion
• func Eventually[T any](tCtx TContext, cb func(TContext) T) gomega.AsyncAssertion
• func SetDefaultVerbosity(v int)
• func Step(tCtx TContext, what string, cb func(tCtx TContext))
• type ContextTB
• type FailureError
  • func (f FailureError) Backtrace() string
  • func (f FailureError) Error() string
  • func (f FailureError) Is(target error) bool
• type InitOption
• type TB
• type TContext
  • func Init(tb TB, opts ...InitOption) TContext
  • func InitCtx(ctx context.Context, tb TB, _ ...InitOption) TContext
  • func NewTestContext(tb testing.TB) (klog.Logger, TContext)
  • func WithCancel(tCtx TContext) TContext
  • func WithClients(tCtx TContext, cfg *rest.Config, ...) TContext
  • func WithContext(parentCtx TContext, ctx context.Context) TContext
  • func WithError(tCtx TContext, err *error) (TContext, func())
  • func WithLogger(tCtx TContext, logger klog.Logger) TContext
  • func WithRESTConfig(tCtx TContext, cfg *rest.Config) TContext
  • func WithStep(tCtx TContext, what string) TContext
  • func WithTB(parentCtx TContext, tb TB) TContext
  • func WithTimeout(tCtx TContext, timeout time.Duration, timeoutCause string) TContext
  • func WithValue(parentCtx TContext, key, val any) TContext
• type Underlier

Constants

const CleanupGracePeriod = 5 * time.Second

CleanupGracePeriod is the time that a TContext gets canceled before the deadline of its underlying test suite (usually determined via "go test -timeout"). This gives the running test(s) time to fail with an informative timeout error. After that, all cleanup callbacks then have the remaining time to complete before the test binary is killed.

For this to work, each blocking calls in a test must respect the cancellation of the TContext.

When using Ginkgo to manage the test suite and running tests, the CleanupGracePeriod is ignored because Ginkgo itself manages timeouts.

Variables

var ErrFailure error = FailureError{}

ErrFailure is an empty error that can be wrapped to indicate that an error is a FailureError. It can also be used to test for a FailureError:.

return fmt.Errorf("some problem%w", ErrFailure)
...
err := someOperation()
if errors.Is(err, ErrFailure) {
    ...
}

Functions

func Consistently

func Consistently[T any](tCtx TContext, cb func(TContext) T) gomega.AsyncAssertion

Consistently wraps gomega.Consistently the same way as Eventually wraps gomega.Eventually.

func Eventually

func Eventually[T any](tCtx TContext, cb func(TContext) T) gomega.AsyncAssertion

Eventually wraps gomega.Eventually such that a failure will be reported via TContext.Fatal.

In contrast to gomega.Eventually, the parameter is strongly typed. It must accept a TContext as first argument and return one value, the one which is then checked with the matcher.

In contrast to direct usage of gomega.Eventually, make additional assertions inside the callback is okay as long as they use the TContext that is passed in. For example, errors can be checked with ExpectNoError:

cb := func(func(tCtx ktesting.TContext) int {
    value, err := doSomething(...)
    tCtx.ExpectNoError(err, "something failed")
    assert(tCtx, 42, value, "the answer")
    return value
}
tCtx.Eventually(cb).Should(gomega.Equal(42), "should be the answer to everything")

If there is no value, then an error can be returned:

cb := func(func(tCtx ktesting.TContext) error {
    err := doSomething(...)
    return err
}
tCtx.Eventually(cb).Should(gomega.Succeed(), "foobar should succeed")

The default Gomega poll interval and timeout are used. Setting a specific timeout may be useful:

tCtx.Eventually(cb).Timeout(5 * time.Second).Should(gomega.Succeed(), "foobar should succeed")

Canceling the context in the callback only affects code in the callback. The context passed to Eventually is not getting canceled. To abort polling immediately because the expected condition is known to not be reached anymore, use gomega.StopTrying:

cb := func(func(tCtx ktesting.TContext) int {
    value, err := doSomething(...)
    if errors.Is(err, SomeFinalErr) {
        // This message completely replaces the normal
        // failure message and thus should include all
        // relevant information.
        //
        // github.com/onsi/gomega/format is a good way
        // to format arbitrary data. It uses indention
        // and falls back to YAML for Kubernetes API
        // structs for readability.
        gomega.StopTrying("permanent failure, last value:\n%s", format.Object(value, 1 /* indent one level */)).
            Wrap(err).Now()
    }
    ktesting.ExpectNoError(tCtx, err, "something failed")
    return value
}
tCtx.Eventually(cb).Should(gomega.Equal(42), "should be the answer to everything")

To poll again after some specific timeout, use gomega.TryAgainAfter. This is particularly useful in Consistently to ignore some intermittent error.

cb := func(func(tCtx ktesting.TContext) int {
    value, err := doSomething(...)
    var intermittentErr SomeIntermittentError
    if errors.As(err, &intermittentErr) {
        gomega.TryAgainAfter(intermittentErr.RetryPeriod).Wrap(err).Now()
    }
    ktesting.ExpectNoError(tCtx, err, "something failed")
    return value
 }
 tCtx.Eventually(cb).Should(gomega.Equal(42), "should be the answer to everything")

func SetDefaultVerbosity

func SetDefaultVerbosity(v int)

SetDefaultVerbosity can be called during init to modify the default log verbosity of the program.

func Step

func Step(tCtx TContext, what string, cb func(tCtx TContext))

Step is useful when the context with the step information is used more than once:

ktesting.Step(tCtx, "step 1", func(tCtx ktesting.TContext) {
 tCtx.Log(...)
    if (... ) {
       tCtx.Failf(...)
    }
)}

Inside the callback, the tCtx variable is the one where the step has been added. This avoids the need to introduce multiple different context variables and risk of using the wrong one.

Types

type ContextTB

type ContextTB interface {
	TB
	CleanupCtx(func(ctx context.Context))
}

ContextTB adds support for cleanup callbacks with explicit context parameter. This is used when integrating with Ginkgo: then CleanupCtx gets implemented via ginkgo.DeferCleanup.

type FailureError

type FailureError struct {
	Msg            string
	FullStackTrace string
}

FailureError is an error where the error string is meant to be passed to [TContext.Fatal] directly, i.e. adding some prefix like "unexpected error" is not necessary. It is also not necessary to dump the error struct.

func (FailureError) Backtrace

func (f FailureError) Backtrace() string

func (FailureError) Error

func (f FailureError) Error() string

func (FailureError) Is

func (f FailureError) Is(target error) bool

type InitOption

type InitOption = initoption.InitOption

type TB

type TB interface {
	Cleanup(func())
	Error(args ...any)
	Errorf(format string, args ...any)
	Fail()
	FailNow()
	Failed() bool
	Fatal(args ...any)
	Fatalf(format string, args ...any)
	Helper()
	Log(args ...any)
	Logf(format string, args ...any)
	Name() string
	Setenv(key, value string)
	Skip(args ...any)
	SkipNow()
	Skipf(format string, args ...any)
	Skipped() bool
	TempDir() string
}

TB is the interface common to testing.T, testing.B, testing.F and github.com/onsi/ginkgo/v2. In contrast to testing.TB, it can be implemented also outside of the testing package.

type TContext

type TContext interface {
	context.Context
	TB

	// Cancel can be invoked to cancel the context before the test is completed.
	// Tests which use the context to control goroutines and then wait for
	// termination of those goroutines must call Cancel to avoid a deadlock.
	//
	// The cause, if non-empty, is turned into an error which is equivalend
	// to context.Canceled. context.Cause will return that error for the
	// context.
	Cancel(cause string)

	// Cleanup registers a callback that will get invoked when the test
	// has finished. Callbacks get invoked in last-in-first-out order (LIFO).
	//
	// Beware of context cancellation. The following cleanup code
	// will use a canceled context, which is not desirable:
	//
	//    tCtx.Cleanup(func() { /* do something with tCtx */ })
	//    tCtx.Cancel()
	//
	// A safer way to run cleanup code is:
	//
	//    tCtx.CleanupCtx(func (tCtx ktesting.TContext) { /* do something with cleanup tCtx */ })
	Cleanup(func())

	// CleanupCtx is an alternative for Cleanup. The callback is passed a
	// new TContext with the same logger and clients as the one CleanupCtx
	// was invoked for.
	CleanupCtx(func(TContext))

	// Expect wraps [gomega.Expect] such that a failure will be reported via
	// [TContext.Fatal]. As with [gomega.Expect], additional values
	// may get passed. Those values then all must be nil for the assertion
	// to pass. This can be used with functions which return a value
	// plus error:
	//
	//     myAmazingThing := func(int, error) { ...}
	//     tCtx.Expect(myAmazingThing()).Should(gomega.Equal(1))
	Expect(actual interface{}, extra ...interface{}) gomega.Assertion

	// ExpectNoError asserts that no error has occurred.
	//
	// As in [gomega], the optional explanation can be:
	//   - a [fmt.Sprintf] format string plus its argument
	//   - a function returning a string, which will be called
	//     lazy to construct the explanation if needed
	//
	// If an explanation is provided, then it replaces the default "Unexpected
	// error" in the failure message. It's combined with additional details by
	// adding a colon at the end, as when wrapping an error. Therefore it should
	// not end with a punctuation mark or line break.
	//
	// Using ExpectNoError instead of the corresponding Gomega or testify
	// assertions has the advantage that the failure message is short (good for
	// aggregation in https://go.k8s.io/triage) with more details captured in the
	// test log output (good when investigating one particular failure).
	ExpectNoError(err error, explain ...interface{})

	// Logger returns a logger for the current test. This is a shortcut
	// for calling klog.FromContext.
	//
	// Output emitted via this logger and the TB interface (like Logf)
	// is formatted consistently. The TB interface generates a single
	// message string, while Logger enables structured logging and can
	// be passed down into code which expects a logger.
	//
	// To skip intermediate helper functions during stack unwinding,
	// TB.Helper can be called in those functions.
	Logger() klog.Logger

	// TB returns the underlying TB. This can be used to "break the glass"
	// and cast back into a testing.T or TB. Calling TB is necessary
	// because TContext wraps the underlying TB.
	TB() TB

	// RESTConfig returns a config for a rest client with the UserAgent set
	// to include the current test name or nil if not available. Several
	// typed clients using this config are available through [Client],
	// [Dynamic], [APIExtensions].
	RESTConfig() *rest.Config

	RESTMapper() *restmapper.DeferredDiscoveryRESTMapper
	Client() clientset.Interface
	Dynamic() dynamic.Interface
	APIExtensions() apiextensions.Interface
}

TContext combines context.Context, TB and some additional methods. Log output is associated with the current test. Errors ([Error], [Errorf]) are recorded with "ERROR" as prefix, fatal errors ([Fatal], [Fatalf]) with "FATAL ERROR".

TContext provides features offered by Ginkgo also when using normal Go testing:

• The context contains a deadline that expires soon enough before the overall timeout that cleanup code can still run.
• Cleanup callbacks can get their own, separate contexts when registered via [CleanupCtx].
• CTRL-C aborts, prints a progress report, and then cleans up before terminating.
• SIGUSR1 prints a progress report without aborting.

Progress reporting is more informative when doing polling with gomega.Eventually and gomega.Consistently. Without that, it can only report which tests are active.

func Init

func Init(tb TB, opts ...InitOption) TContext

Init can be called in a unit or integration test to create a test context which: - has a per-test logger with verbosity derived from the -v command line flag - gets canceled when the test finishes (via [TB.Cleanup])

Note that the test context supports the interfaces of TB and context.Context and thus can be used like one of those where needed. It also has additional methods for retrieving the logger and canceling the context early, which can be useful in tests which want to wait for goroutines to terminate after cancellation.

If the TB implementation also implements ContextTB, then [TContext.CleanupCtx] uses [ContextTB.CleanupCtx] and uses the context passed into that callback. This can be used to let Ginkgo create a fresh context for cleanup code.

Can be called more than once per test to get different contexts with independent cancellation. The default behavior describe above can be modified via optional functional options defined in initoption.

func InitCtx

func InitCtx(ctx context.Context, tb TB, _ ...InitOption) TContext

InitCtx is a variant of Init which uses an already existing context and whatever logger and timeouts are stored there. Functional options are part of the API, but currently there are none which have an effect.

func NewTestContext

func NewTestContext(tb testing.TB) (klog.Logger, TContext)

NewTestContext is a replacement for ktesting.NewTestContext which returns a more versatile context.

func WithCancel

func WithCancel(tCtx TContext) TContext

WithCancel sets up cancellation in a [TContext.Cleanup] callback and constructs a new TContext where [TContext.Cancel] cancels only the new context.

func WithClients

func WithClients(tCtx TContext, cfg *rest.Config, mapper *restmapper.DeferredDiscoveryRESTMapper, client clientset.Interface, dynamic dynamic.Interface, apiextensions apiextensions.Interface) TContext

WithClients uses an existing config and clients.

func WithContext

func WithContext(parentCtx TContext, ctx context.Context) TContext

WithContext constructs a new TContext with a different Context instance. This can be used in callbacks which receive a Context, for example from Gomega:

gomega.Eventually(tCtx, func(ctx context.Context) {
   tCtx := ktesting.WithContext(tCtx, ctx)
   ...

This is important because the Context in the callback could have a different deadline than in the parent TContext.

func WithError

func WithError(tCtx TContext, err *error) (TContext, func())

WithError creates a context where test failures are collected and stored in the provided error instance when the caller is done. Use it like this:

func doSomething(tCtx ktesting.TContext) (finalErr error) {
     tCtx, finalize := WithError(tCtx, &finalErr)
     defer finalize()
     ...
     tCtx.Fatal("some failure")

Any error already stored in the variable will get overwritten by finalize if there were test failures, otherwise the variable is left unchanged. If there were multiple test errors, then the error will wrap all of them with errors.Join.

Test failures are not propagated to the parent context.

func WithLogger

func WithLogger(tCtx TContext, logger klog.Logger) TContext

WithLogger constructs a new context with a different logger.

func WithRESTConfig

func WithRESTConfig(tCtx TContext, cfg *rest.Config) TContext

WithRESTConfig initializes all client-go clients with new clients created for the config. The current test name gets included in the UserAgent.

func WithStep

func WithStep(tCtx TContext, what string) TContext

WithStep creates a context where a prefix is added to all errors and log messages, similar to how errors are wrapped. This can be nested, leaving a trail of "bread crumbs" that help figure out where in a test some problem occurred or why some log output gets written:

ERROR: bake cake: set heat for baking: oven not found

The string should describe the operation that is about to happen ("starting the controller", "list items") or what is being operated on ("HTTP server"). Multiple different prefixes get concatenated with a colon.

func WithTB

func WithTB(parentCtx TContext, tb TB) TContext

WithTB constructs a new TContext with a different TB instance. This can be used to set up some of the context, in particular clients, in the root test and then run sub-tests:

func TestSomething(t *testing.T) {
   tCtx := ktesting.Init(t)
   ...
   tCtx = ktesting.WithRESTConfig(tCtx, config)

   t.Run("sub", func (t *testing.T) {
       tCtx := ktesting.WithTB(tCtx, t)
       ...
   })

WithTB sets up cancellation for the sub-test.

func WithTimeout

func WithTimeout(tCtx TContext, timeout time.Duration, timeoutCause string) TContext

WithTimeout sets up new context with a timeout. Canceling the timeout gets registered in a [TContext.Cleanup] callback. [TContext.Cancel] cancels only the new context. The cause is used as reason why the context is canceled once the timeout is reached. It may be empty, in which case the usual "context canceled" error is used.

func WithValue

func WithValue(parentCtx TContext, key, val any) TContext

WithValue wraps context.WithValue such that the result is again a TContext.

type Underlier

type Underlier = ktesting.Underlier

Underlier is the additional interface implemented by the per-test LogSink behind [TContext.Logger]. Together with initoption.BufferLogs it can be used to capture log output in memory to check it in tests.