stepwise

README

Stepwise

Vault testing Stepwise was cloned from https://github.com/hashicorp/vault-testing-stepwise.

It provides a mini framework for writing acceptance tests for custom Vault plugins.

Usage

The framework is simple, set the plugin name, which is used for building and mounting the plugin. Use that to create a new environment, and add test steps

package pluginname

import (
	"os"
	"testing"

	stepwise "github.com/CSCfi/vault-testing-stepwise"
	"github.com/CSCfi/vault-testing-stepwise/environments/docker"
)

func TestPlugin(t *testing.T) {
	err := os.Setenv("VAULT_ACC", "1")
	if err != nil {
		t.Error("Failed to set environment variable VAULT_ACC")
	}
	
	// The mount options are also used for locating and building the plugin
	mountOptions := stepwise.MountOptions{
		MountPathPrefix: "plugin-mount-prefix",
		RegistryName:    "plugin-name-registry",
		PluginType:      stepwise.PluginTypeHere,
		PluginName:      "plugin-name",
	}
	keysCase := stepwise.Case{
		Environment:  docker.NewEnvironment("DockerPluginEnvironment", &mountOptions, "hashicorp/vault:latest"),
		SkipTeardown: false,
		Steps: []stepwise.Step{
			stepwise.Step{
				Name:      "testThatPathWrites",
				Operation: stepwise.WriteOperation,
				Data:      map[string]interface{}{"field": "value"},
				// Instead of passing static data, it's possible to use GetData with a function that returns the data    
				Path:      "/path/here",
			},
			stepwise.Step{
                Name:      "testThatPathReturnsX",
                Operation: stepwise.ReadOperation,
                Path:      "/path/here",
				// There's support for ReadOperation with data by passing ReadData 
                Assert: func (resp *api.Secret, err error) error {
                
                    // resp.Data contains the `data` part of the response from Vault
                    
                    return nil
                },
            },
		},
	}
	
    // Running the case compiles the plugin with Docker, and runs Vault with the plugin enabled.
    // Each step in a case is run sequentially.
    // At the end of the case, the Docker container and network are removed, unless `SkipTeardown` is set to `true`
    stepwise.Run(t, keysCase)
}

Debugging

Set SkipTeardown to true, and go into the container with docker exec -it container-id sh. Find the container ID with docker ps.

When leaving the container running, it's possible to get the root token, and make API calls directly to Vault. For that, don't execute stepwise.Run instead, create an environment, run it's Setup() function, and print the env.RootToken() to the command line.

Tests

Can be executed with

go test -v ./...

Licensing

vault-testing-stepwise is licensed under MPL-2.0.

SPDX-License-Identifier: MPL-2.0

Documentation

Overview

Package stepwise offers types and functions to enable black-box style tests that are executed in defined set of steps. Stepwise utilizes "Environments" which setup a running instance of Vault and provide a valid API client to execute user defined steps against.

Index

• Constants
• func CompilePlugin(name, pluginName, srcDir, tmpDir string) (string, string, string, error)
• func Run(tt TestT, c Case)
• type AssertionFunc
• type Case
• type CertificateGetter
  • func NewCertificateGetter(certFile, keyFile, passphrase string) *CertificateGetter
  • func (cg *CertificateGetter) GetCertificate(clientHello *tls.ClientHelloInfo) (*tls.Certificate, error)
  • func (cg *CertificateGetter) Reload() error
• type Environment
• type MountOptions
• type Operation
• type PluginType
  • func (p PluginType) String() string
• type ReloadFunc
• type Step
• type TestT

Constants

const TestEnvVar = "VAULT_ACC"

TestEnvVar must be set to a non-empty value for acceptance tests to run.

Functions

func CompilePlugin

func CompilePlugin(name, pluginName, srcDir, tmpDir string) (string, string, string, error)

CompilePlugin is a helper method to compile a source plugin

func Run

func Run(tt TestT, c Case)

Run performs an acceptance test on a backend with the given test case.

Tests are not run unless an environmental variable "VAULT_ACC" is set to some non-empty value. This is to avoid test cases surprising a user by creating real resources.

Tests will fail unless the verbose flag (`go test -v`, or explicitly the "-test.v" flag) is set. Because some acceptance tests take quite long, we require the verbose flag so users are able to see progress output.

Types

type AssertionFunc

type AssertionFunc func(*api.Secret, error) error

AssertionFunc is the callback used for Assert in Steps.

type Case

type Case struct {
	// Environment is used to setup the Vault instance and provide the client that
	// will be used to drive the tests
	Environment Environment

	// Precheck enabls a test case to determine if it should run or not
	Precheck func()

	// Steps are the set of operations that are run for this test case. During
	// execution each step will be logged to output with a 1-based index as it is
	// ran, with the first step logged as step '1' and not step '0'.
	Steps []Step

	// SkipTeardown allows the Environment TeardownFunc to be skipped, leaving any
	// infrastructure created after the test exists. This is useful for debugging
	// during plugin development to examine the state of the Vault cluster after a
	// test runs. Depending on the Environment used this could incur costs the
	// user is responsible for.
	SkipTeardown bool
}

Case represents a scenario we want to test which involves a series of steps to be followed sequentially, evaluating the results after each step.

type CertificateGetter

type CertificateGetter struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

CertificateGetter satisfies ReloadFunc and its GetCertificate method satisfies the tls.GetCertificate function signature. Currently it does not allow changing paths after the fact.

func NewCertificateGetter

func NewCertificateGetter(certFile, keyFile, passphrase string) *CertificateGetter

func (*CertificateGetter) GetCertificate

func (cg *CertificateGetter) GetCertificate(clientHello *tls.ClientHelloInfo) (*tls.Certificate, error)

func (*CertificateGetter) Reload

func (cg *CertificateGetter) Reload() error

type Environment

type Environment interface {
	// Setup is responsible for creating the Vault cluster for use in the test
	// case.
	Setup() error

	// Client should return a clone of a configured Vault API client to
	// communicate with the Vault cluster created in Setup and managed by this
	// Environment.
	Client() (*api.Client, error)

	// Teardown is responsible for destroying any and all infrastructure created
	// during Setup or otherwise over the course of executing test cases.
	Teardown() error

	// Name returns the name of the environment provider, e.g. Docker, Minikube,
	// et.al.
	Name() string

	// MountPath returns the path the plugin is mounted at
	MountPath() string

	// RootToken returns the root token of the cluster, used for making requests
	// as well as administrative tasks
	RootToken() string
}

Environment is the interface Environments need to implement to be used in Case to execute each Step

type MountOptions

type MountOptions struct {
	// MountPathPrefix is an optional prefix to use when mounting the plugin. If
	// omitted the mount path will default to the PluginName with a random suffix.
	MountPathPrefix string

	// Name is used to register the plugin. This can be arbitrary but should be a
	// reasonable value. For an example, if the plugin in test is a secret backend
	// that generates UUIDs with the name "vault-plugin-secrets-uuid", then "uuid"
	// or "test-uuid" would be reasonable. The name is used for lookups in the
	// catalog. See "name" in the "Register Plugin" endpoint docs:
	// - https://www.vaultproject.io/api-docs/system/plugins-catalog#register-plugin
	RegistryName string

	// PluginType is the optional type of plugin. See PluginType const defined
	// above
	PluginType api.PluginType

	// PluginName represents the name of the plugin that gets compiled. In the
	// standard plugin project file layout, it represents the folder under the
	// cmd/ folder. In the below example UUID project, the PluginName would be
	// "uuid":
	//
	// vault-plugin-secrets-uuid/
	// - backend.go
	// - cmd/
	// ----uuid/
	// ------main.go
	// - path_generate.go
	//
	PluginName string
}

MountOptions are a collection of options each step driver should support

type Operation

type Operation string

Operation defines operations each step could perform. These are intentionally redefined from the logical package in the SDK, so users consistently use the stepwise package and not a combination of both stepwise and logical.

const (
	WriteOperation  Operation = "create"
	UpdateOperation Operation = "update"
	ReadOperation   Operation = "read"
	DeleteOperation Operation = "delete"
	ListOperation   Operation = "list"
	HelpOperation   Operation = "help"
)

type PluginType

type PluginType consts.PluginType

PluginType defines the types of plugins supported This type re-create constants as a convenience so users don't need to import/use the consts package.

const (
	PluginTypeUnknown PluginType = iota
	PluginTypeCredential
	PluginTypeDatabase
	PluginTypeSecrets
)

These are originally defined in sdk/helper/consts/plugin_types.go

func (PluginType) String

func (p PluginType) String() string

type ReloadFunc

type ReloadFunc func() error

ReloadFunc are functions that are called when a reload is requested

type Step

type Step struct {

	// Name of the step to be printed in the report
	Name string

	// Operation defines what action is being taken in this step; write, read,
	// delete, et. al.
	Operation Operation

	// Path is the localized request path. The mount prefix, namespace, and
	// optionally "auth" will be automatically added.
	Path string

	// Arguments to pass in the request. These arguments represent payloads sent
	// to the API.
	Data map[string]interface{}

	// Alternatively get data from a function
	GetData func() (map[string]interface{}, error)

	// BodyData is the data to pass to a read or delete request
	BodyData map[string][]string

	// Assert is a function that is called after this step is executed in order to
	// test that the step executed successfully. If this is not set, then the next
	// step will be called
	Assert AssertionFunc

	// Unauthenticated will make the request unauthenticated.
	Unauthenticated bool
}

Step represents a single step of a test Case

type TestT

type TestT interface {
	Error(args ...interface{})
	Fatal(args ...interface{})
	Skip(args ...interface{})
	Helper()
}

TestT is the interface used to handle the test lifecycle of a test.

Users should just use a *testing.T object, which implements this.