copper

README

Copper

Copper keeps REST APIs honest

Introduction

Copper is a test library intended to verify that API contracts are upheld when exercising a REST API from a test. Copper will make sure that all the endpoints in your OpenAPI specification are tested, and that bodies, paths, parameters, and response codes exchanged between the test client and server (system under test) are correct. Copper will also fail the test if undocumented endpoints are being visited.

Why?

API contracts and documentation have always been a good way to communicate intent between a backend API and client code. However, throughout the life-time of a project, things happen:

• Endpoints change body payloads, parameters, or responses.
• Endpoints are added, but documentation is forgotten.
• OpenAPI specification becomes invalid (was it ever actually valid?)
• Test coverage is missing.

All of the above points contribute to untrustworthy specifications, which in turn seriously reduces their effectiveness as a tool of communication and reference.

Copper resolves all of these pain points in a non-intrusive way by verifying interactions in the test phase. This means the APIs can stay lean during operation, while confidence in both documentation and implementation can remain high.

100% test coverage

Copper enforces 100% test coverage of all declared paths, methods, and response codes. This does not mean that the application will have a 100% line coverage, which in general is seen as overkill and both impractical and having low return on investment. Having a 100% coverage of API endpoints, however, allows for the important main paths of the API to be verified in tests, and Copper provides an automated check for that this is indeed happening.

Having full test coverage of the API also helps retain backwards compatibility of changes by creating a circular dependency between the tests and the specification. This also makes sure that any new endpoints have to be both documented and tested, or neither.

NOTE: This behaviour can be disabled with an option.

Usage

Copper is used from integration/contract style tests. Wrap the HTTP client being used in copper and then use that client for performing the API calls in your test case. This works best from a single main test for a single spec, and then subtest for the specific endpoints/use cases.

func TestVersion1(t *testing.T) {
	f, err := os.Open("spec.yaml")
	if err != nil {
		t.Fatal(err)
	}
	defer f.Close()

	client, err := copper.WrapClient(http.DefaultClient, f)
	if err != nil {
		t.Fatal(err)
	}

	t.Run("ping", func(t *testing.T) {
		testPingEndpoints(t, client)
	}
	t.Run("other use case that I have", func(t *testing.T) {
		testMyOtherThing(t, client)
	}
	
	// Verifying at the end checks that all paths, methods and responses are covered and that no extra paths have been hit.
	client.Verify(t)
}

See the examples for complete examples.

Options

To alter the behavior of copper and control what type of validation will be done, functional options can be passed to the WrapClient or stand-alone NewVerifier constructors. The options are as follows:

• WithServer: Sets the server and base path for the API to allow the spec to be mapped to the actual server endpoints. This is especially useful if the spec doesn't contain a server with the base path that the target of the tests have, or if the wrong entry is being used for verification. Without this option, the first server in the list that matches the base path of the request will be used.
• WithInternalServerErrors: Also verify that all declared 500 responses have been tested. This is not really recommended since if an internal server error can be produced in a test, the problem should probably just be fixed instead.
• WithRequestValidation: Also validate that the request adheres to the spec. This can be useful when developing the tests as it checks that the client is well-behaved, but makes less sense once the contract tests are done, as the server should ideally be lenient in the data that it accepts.
• WithoutFullCoverage: Do not require full coverage of all methods, paths and response codes.

Building

As Copper is a library, it will not build into a standalone binary. Copper is a standard go project, and only needs the go tooling to test:

go vet ./... 
go test ./...

Documentation

Index

• Variables
• type Endpoint
• type Option
  • func WithInternalServerErrors() Option
  • func WithRequestLogging(l RequestLogger) Option
  • func WithRequestValidation() Option
  • func WithServer(host string) Option
  • func WithoutFullCoverage() Option
• type RequestLogger
• type SentinelError
  • func (s SentinelError) Error() string
• type ValidatingClient
  • func WrapClient(c *http.Client, spec io.Reader, opts ...Option) (*ValidatingClient, error)
  • func (v *ValidatingClient) Delete(url string) (resp *http.Response, err error)
  • func (v *ValidatingClient) Do(r *http.Request) (*http.Response, error)
  • func (v *ValidatingClient) Get(url string) (resp *http.Response, err error)
  • func (v *ValidatingClient) Head(url string) (resp *http.Response, err error)
  • func (v *ValidatingClient) Post(url string, contentType string, body io.Reader) (resp *http.Response, err error)
  • func (v *ValidatingClient) Put(url string, contentType string, body io.Reader) (resp *http.Response, err error)
  • func (v *ValidatingClient) WithClient(c *http.Client) (*ValidatingClient, error)
• type VerificationError
  • func (v *VerificationError) Error() string
  • func (v *VerificationError) Sentinel() error
  • func (v *VerificationError) Unwrap() []error
• type Verifier
  • func NewVerifier(specBytes []byte, opts ...Option) (*Verifier, error)
  • func (v *Verifier) CurrentError() error
  • func (v *Verifier) CurrentErrors() []error
  • func (v *Verifier) Record(res *http.Response)
  • func (v *Verifier) Reset()
  • func (v *Verifier) Verify(t *testing.T)

Variables

var (
	ErrNotChecked      = SentinelError{"not checked"}
	ErrNotPartOfSpec   = SentinelError{"not part of spec"}
	ErrResponseInvalid = SentinelError{"response invalid"}
	ErrRequestInvalid  = SentinelError{"request invalid"}
)

Types

type Endpoint

type Endpoint struct {
	Path         string
	Method       string
	ResponseCode string
}

Endpoint represents a single coordinate in the endpoints tree.

type Option

type Option func(c *config)

func WithInternalServerErrors

func WithInternalServerErrors() Option

WithInternalServerErrors is a functional Option for also validating server responses. These are skipped by default since a server should not ideally have internal server errors, and even if they are not part of a specification, they considered a possible response from an API.

func WithRequestLogging

func WithRequestLogging(l RequestLogger) Option

WithRequestLogging is a functional Option that provides a logger that copper will use to log out requests and responses. This can be useful for debugging, or writing initial tests for an endpoint, but will add quite a lot of log output for larger test suites.

func WithRequestValidation

func WithRequestValidation() Option

WithRequestValidation is a functional Option for checking request parameters and bodies as they are sent. Doing validation of the request by default might conflict with checking error cases (400 responses specifically), so it does not happen by default. Enabling checking will produce an error for each request that is not in accordance with the specification for that endpoint.

func WithServer

func WithServer(host string) Option

WithServer is a functional Option for setting the base path/host used when correlating the specification to the API calls being recorded. This can be used when the specification doesn't have a server entry for the target of tests, or when conflicts cause the wrong server to be selected for calculating the base path.

func WithoutFullCoverage

func WithoutFullCoverage() Option

WithoutFullCoverage is a functional Option for disabling verification that full coverage of the API has been accomplished. Full coverage is defined as having a test covering all documented response codes for all documented endpoint paths and methods. Using this option will still verify that no undocumented endpoints have been hit, as well as checking schemas for all valid interactions.

type RequestLogger

type RequestLogger interface {
	Logf(format string, args ...any)
}

RequestLogger is a minimal interface that can fit for example a testing.T, allowing tests to easily print logs where needed.

type SentinelError

type SentinelError struct {
	// contains filtered or unexported fields
}

func (SentinelError) Error

func (s SentinelError) Error() string

type ValidatingClient

type ValidatingClient struct {
	*Verifier
	// contains filtered or unexported fields
}

ValidatingClient provides an HTTP client, and wraps the main methods, recording any and all paths that are being called.

func WrapClient

func WrapClient(c *http.Client, spec io.Reader, opts ...Option) (*ValidatingClient, error)

WrapClient takes an HTTP client and io.Reader for the OpenAPI spec. The spec is parsed, and wraps the client so that the outbound calls are now recorded when made.

func (*ValidatingClient) Delete

func (v *ValidatingClient) Delete(url string) (resp *http.Response, err error)

Delete records response for HTTP DELETE requests

func (*ValidatingClient) Do

func (v *ValidatingClient) Do(r *http.Request) (*http.Response, error)

Do takes any http.Request, sends it to the server it and then records the result.

func (*ValidatingClient) Get

func (v *ValidatingClient) Get(url string) (resp *http.Response, err error)

Get is a convenience method for recording responses for HTTP GET requests

func (*ValidatingClient) Head

func (v *ValidatingClient) Head(url string) (resp *http.Response, err error)

Head is a convenience method for recording responses for HTTP HEAD requests

func (*ValidatingClient) Post

func (v *ValidatingClient) Post(url string, contentType string, body io.Reader) (resp *http.Response, err error)

Post is a convenience method for recording responses for HTTP POST requests

func (*ValidatingClient) Put

func (v *ValidatingClient) Put(url string, contentType string, body io.Reader) (resp *http.Response, err error)

Put is a convenience method for recording responses for HTTP PUT requests

func (*ValidatingClient) WithClient

func (v *ValidatingClient) WithClient(c *http.Client) (*ValidatingClient, error)

WithClient returns a new client using the same validator, but a new client. This can be useful to change transport or authorization settings, while still contributing to the same spec validation.

type VerificationError

type VerificationError struct {
	// contains filtered or unexported fields
}

func (*VerificationError) Error

func (v *VerificationError) Error() string

func (*VerificationError) Sentinel

func (v *VerificationError) Sentinel() error

func (*VerificationError) Unwrap

func (v *VerificationError) Unwrap() []error

type Verifier

type Verifier struct {
	// contains filtered or unexported fields
}

func NewVerifier

func NewVerifier(specBytes []byte, opts ...Option) (*Verifier, error)

NewVerifier takes bytes for an OpenAPI spec and options, and then returns a new Verifier for the given spec. Supply zero or more Option instances to change the behaviour of the Verifier.

func (*Verifier) CurrentError

func (v *Verifier) CurrentError() error

CurrentError is a convenience method for CurrentErrors, where the errors are joined into a single error, making it easier to check.

func (*Verifier) CurrentErrors

func (v *Verifier) CurrentErrors() []error

CurrentErrors return the current collection of errors in the verifier.

func (*Verifier) Record

func (v *Verifier) Record(res *http.Response)

func (*Verifier) Reset

func (v *Verifier) Reset()

Reset will remove all current errors, and start the Verifier from scratch. This allows it to be reused.

func (*Verifier) Verify

func (v *Verifier) Verify(t *testing.T)

Verify will cause the given test context to fail with an error if Error returns a non-nil error.