http

README

blugnu/http

A net/http.Client wrapper with quality of life improvements:

• Configurable request retries
• Simplified response handling
• Multipart form data transformation to/from maps
• JSON marshalling helpers for request and response bodies
• A mock client for request and response mocking

Installation

go get github.com/blugnu/http

Using the Client

The NewClient() function in the github.com/blugnu/http package is used to create a new http.Client:

param	type	description
name	string	a name for the client, used in error messages and test failure reports
url	string	the base url for the client
opts	...ClientOption	optional client configuration

The function returns an HttpClient interface providing the following methods:

method	description
NewRequest(ctx context.Context, method string, path string, opts ...RequestOption) (*http.Request, error)	creates a new request with the specified method and path, and additional request options as specified
Delete(ctx context.Context, url string, opts ...RequestOption) (*http.Response, error)	performs a DELETE request using a specified path and request options as specified
Get(ctx context.Context, url string, opts ...RequestOption) (*http.Response, error)	performs a GET request using a specified path and request options as specified
Patch(ctx context.Context, url string, opts ...RequestOption) (*http.Response, error)	performs a PATCH request using a specified path and request options as specified
Post(ctx context.Context, url string, opts ...RequestOption) (*http.Response, error)	performs a POST request using a specified path and request options as specified
Put(ctx context.Context, url string, opts ...RequestOption) (*http.Response, error)	performs a PUT request using a specified path and request options as specified
Do(rq *http.Request) (*http.Response, error)	performs a request using the specified http.Request, initialised separately

Response Handling

The client in this module provides extended handling of responses, to simplify error handling in the code using the client. In addition to any error that might result from attempting to perform the request, the following additional errors may also be returned with or without a response:

error	response included	description
http.ErrNoResponseBody	yes	returned if the response body is empty and the request.ResponseBodyRequired() request option was specified; NOTE: will never be returned if request.StreamResponse() is also specified
http.ErrUnexpectedStatusCode	yes	returned if the response has a status code other than http.StatusOK and which is not identified as acceptable using the request.AcceptStatus() request option
http.ErrMaxRetriesExceeded	no	returned if the request was retried the maximum number of times specified for the request

Maximum retries for a request are determined by the request.MaxRetries() request option or a http.MaxRetries client option configured on the client used to make the request. When a http.ErrMaxRetriesExceeded error is returned it is wrapped with the error that occurred returned when making the final, failed request

Acceptable Status Codes

By default, the only acceptable status code for a response is http.StatusOK. A response with any other status code will result in an http.ErrUnexpectedStatusCode error. This may be overridden using the request.AcceptStatus() request option, which configures the request to treat the specified status code as acceptable.

Examples

: response body is expected

r, err := client.Get(ctx, "v1/customer",
    request.ResponseBodyRequired(),
)
if err != nil {
    return err
}

// ... proceed with processing the response body

: return a specific error when receiving 404 Not Found

r, err := client.Get(ctx, "v1/customer",
    request.AcceptStatus(http.StatusNotFound),
)
if err != nil {
    return err
}
switch {
    case r.StatusCode == http.StatusNotFound:
        return ErrCustomerNotFound

    default:
        // can only be an OK response; client.Get() would otherwise 
        // have returned ErrUnexpectedStatusCode
}

Request Options

Request options are used to configure the properties of a request. The following request options are provided:

option	description
request.Accept()	adds an Accept header to the request
request.AcceptStatus()	configures the request to accept a specific status code
request.BearerToken()	adds an Authorization header with a value of Bearer
request.Body()	adds a body to the request
request.ContentType()	adds a Content-Type header to the request
request.Header()	adds a canonical header to the request
request.JSONBody()	adds a JSON body to the request, marshalling a supplied any
request.MaxRetries()	configures the request to be retried; overrides any retries configured on the client
request.MultipartFormDataFromMap()	adds a multipart form data body to the request
request.NonCanonicalHeader()	adds a non-canonical header to the request
request.Query()	adds a map of query parameters to the request
request.QueryP()	adds an individual key:value parameter to the request query
request.RawQuery()	specifies an appropriately url encoded query string for the request
request.StreamResponse()	configures the response to be streamed

Some of these options can affect the behaviour of the client when processing a response:

option	affect on client
request.AcceptStatus()	prevents the client from returning an error if the response status code is configured as acceptable
request.MaxRetries()	causes the client to retry the request if the response status code is not acceptable; overrides any http.MaxRetries() option if specified on the client used to perform the request
request.ResponseBodyRequired()	causes the client to return an error if the response body is empty; has no effect if request.StreamResponse() is also specified
request.StreamResponse()	causes the response body to be streamed

Multipart Form Data

Requests

To submit a multipart form data body with a request, the request.MultipartFormDataFromMap() request option may be used.

This is a generic function with type parameters for key and value types in a supplied map. These types will be inferred from a function that must also be provided to be called for each key:value in the map to encode that key:value as an individual part in the form data.

The supplied function must accepts a key and value parameter of the keys and values in the map; the function must return a field name string, filename string and data []byte for each part, or an error.

resp, err := client.Post(ctx, "v1/documents",
        request.MultipartFormDataFromMap(docs, func(id string, doc Document) (string, string, []byte, error) {
            return doc.id, doc.filename, doc.Content, nil
        }),
    )

Responses

When handling responses containing multipart form data, a corresponding function is provided that will parse a response containing a multipart form data body and transform it into a map: MapFromMultipartFormData().

This is again a generic function also accepting a function which in this case performs the transformation in reverse. The function is called with the field name, filename and data for each part in the multipart form and must return a key:value pair to be stored in the map, or an error.

    docs, err := http.MapFromMultipartFormData[string, []byte](ctx, r,
        func(field, filename string, data []byte) (string, []byte, error) {
            return filename, data, nil
        })
    if err != nil {
        return err
    }

---

Mocking

This module provides two facilities for mocking http Client behaviors:

1. testing that code under test issues the expected requests
2. providing mock responses to http requests issues by code under test

Both use cases start with creating a mock client using the NewMockClient() function:

   client, mock := http.NewMockClient("client")

The name argument to the function is used in error messages and test failure reports to identify the client involved.

The client returned from this function should be injected into code under test, to replace the production Client.

The mock returned by the function is used to set and test expected request properties and to establish mock responses for those requests.

Using a Mock to Verify Expected Requests

    mock.ExpectGet("v1/customer")

This configures the mock to expect a GET request to the specified url. With no other configuration specified, any GET request will satisfy this expectation. Normally, specific properties of the expected request will be configured using the fluent api for configuring expected request properties.

For example, if the url involved required an authorization header then it would be typical to specify that the request is expected to include the appropriate header:

    mock.ExpectGet("v1/customer").
        WithHeader("Authorisation")

After the code under test has been executed, the mock may then be used to verify that the expected requests were made with the correct properties using the ExpectationsWereMet() method of the mock. This returns an error describing any expectations that were not satisfied or nil if all expectations were met:

    // ARRANGE
    mock.ExpectGet("v1/customer").
        WithHeader("Authorisation")

    // ACT
    ...

    // ASSERT
    if err := mock.ExpectationsWereMet(); err != nil {
        t.Error(err)
    }

Mocking Responses

If no response details are configured for an expected request, the mock client will provide a 200 OK response with no body or headers.

This is configurable using the fluent api returned by a mocked request to configure the response to be returned.

For example, to mock a 403 Forbidden response:

    mock.ExpectGet("v1/customer").
        WithHeader("Authorisation").
        WillRespond().WithStatusCode(http.StatusForbidden)

To provide more detailed configuration of a response, identifying one or more headers, body and status code details, the WillRespond() method provides a response configuration fluent api:

    mock.ExpectGet("v1/customer").
        WithHeader("Authorisation").
        WillRespond().
            WithHeader("Content-Type", "application/json").
            WithBody([]byte(`{"id":1,"name":"Jane Smith"}`))

Documentation

Index

• Constants
• Variables
• func MapFromMultipartFormData[K comparable, V any](ctx context.Context, r *http.Response, ...) (map[K]V, error)
• func NewMockClient(name string, wrap ...) (HttpClient, MockClient)
• func UnmarshalJSON[T any](ctx context.Context, r *http.Response) (T, error)
• type Client
• type ClientInterface
• type ClientOption
  • func MaxRetries(n uint) ClientOption
  • func URL(u any) ClientOption
  • func Using(httpClient interface{ ... }) ClientOption
• type HttpClient
  • func NewClient(name string, opts ...ClientOption) (HttpClient, error)
• type MockClient
• type MockExpectationsError
  • func (err MockExpectationsError) Error() string
• type MockRequest
  • func (rq MockRequest) String() string
  • func (mock *MockRequest) WillNotBeCalled()
  • func (mock *MockRequest) WillRespond() *mockResponse
  • func (mock *MockRequest) WillReturnError(err error)
  • func (mock *MockRequest) WithBody(b []byte) *MockRequest
  • func (mock *MockRequest) WithHeader(k string, v ...string) *MockRequest
  • func (mock *MockRequest) WithNonCanonicalHeader(k string, v ...string) *MockRequest
• type Request
• type RequestOption
• type Response
• type ResponseWriter
• type RoundTripper
• type Transport

Constants

const (
	MethodConnect = http.MethodConnect
	MethodDelete  = http.MethodDelete
	MethodGet     = http.MethodGet
	MethodHead    = http.MethodHead
	MethodOptions = http.MethodOptions
	MethodPatch   = http.MethodPatch
	MethodPost    = http.MethodPost
	MethodPut     = http.MethodPut
	MethodTrace   = http.MethodTrace
)

const (
	StatusBadRequest          = http.StatusBadRequest
	StatusForbidden           = http.StatusForbidden
	StatusInternalServerError = http.StatusInternalServerError
	StatusNotAcceptable       = http.StatusNotAcceptable
	StatusNotFound            = http.StatusNotFound
	StatusOK                  = http.StatusOK
	StatusUnauthorized        = http.StatusUnauthorized
)

Variables

var (
	NoBody         = http.NoBody
	ListenAndServe = http.ListenAndServe
)

var (
	ErrInitialisingClient   = errors.New("error initialising client")
	ErrInitialisingRequest  = errors.New("error initialising request")
	ErrInvalidJSON          = errors.New("invalid json")
	ErrInvalidRequestHeader = errors.New("invalid request headers")
	ErrInvalidURL           = errors.New("invalid url")
	ErrMaxRetriesExceeded   = errors.New("http retries exceeded")
	ErrNoResponseBody       = errors.New("response body was empty")
	ErrReadingResponseBody  = errors.New("error reading response body")
	ErrUnexpectedStatusCode = errors.New("unexpected status code")

	// errors related to the mock client
	ErrCannotChangeExpectations = errors.New("expectations cannot be changed")
	ErrUnexpectedRequest        = errors.New("unexpected request")
)

Functions

func MapFromMultipartFormData

func MapFromMultipartFormData[K comparable, V any](
	ctx context.Context,
	r *http.Response,
	fn func(string, string, []byte) (K, V, error),
) (map[K]V, error)

MapFromMultipartFormData is a generic function that parses an http.Response body expected to contain multipart form data, transforming each part into a key-value pair using a supplied function.

func NewMockClient

func NewMockClient(name string, wrap ...func(c interface {
	Do(*http.Request) (*http.Response, error)
}) interface {
	Do(*http.Request) (*http.Response, error)
}) (HttpClient, MockClient)

NewMockClient returns a new http.HttpClient to be used for making requests and an http.MockClient on which expected requests, and corresponding responses, may be configured.

params

name          // used to identify the mock client in test failure reports and errors
wrap          // optional function(s) to wrap the client with some other client
              // implementation, if required; nil functions are ignored

returns

HttpClient    // used to make requests; this should be injected into code under test
MockClient    // used to configure expected requests and provide details of responses
              // to be mocked for each request

Note the use of an anonymous interface in the exported function signature. This avoids creating coupling modules thru a shared reference to an interface type. In Go (currently at least) interfaces are fungible but interface types are not.

func UnmarshalJSON

func UnmarshalJSON[T any](ctx context.Context, r *http.Response) (T, error)

UnmarshalJSON is a generic function that unmarshals the body of an http.Response into a value of a specified type.

The function returns an error if the body cannot be read or if the body does not contain valid JSON and the result will be the zero value of the generic type.

Types

type Client

type Client = http.Client

type ClientInterface

type ClientInterface interface {
	Do(*http.Request) (*http.Response, error)
}

ClientInterface is an interface that describes a wrappable http client

type ClientOption

type ClientOption func(*client) error

ClientOption is a function that applies an option to a client

func MaxRetries

func MaxRetries(n uint) ClientOption

MaxRetries sets the maximum number of retries for requests made using the client. Individual requests may be configured to override this value on a case-by-case basis.

func URL

func URL(u any) ClientOption

URL sets the base URL for requests made using the client. The URL may be specified as a string or a *url.URL.

If a string is provided, it will be parsed to ensure it is a valid, absolute URL.

If a URL is provided is must be absolute.

func Using

func Using(httpClient interface {
	Do(*http.Request) (*http.Response, error)
}) ClientOption

Using sets the HTTP client to use for requests made using the client. Any value that implements the `Do(*http.Request) (*http.Response, error)` method may be used.

type HttpClient

type HttpClient interface {
	Delete(context.Context, string, ...RequestOption) (*http.Response, error)
	Do(*http.Request) (*http.Response, error)
	Get(context.Context, string, ...RequestOption) (*http.Response, error)
	Patch(context.Context, string, ...RequestOption) (*http.Response, error)
	Post(context.Context, string, ...RequestOption) (*http.Response, error)
	Put(context.Context, string, ...RequestOption) (*http.Response, error)
	NewRequest(context.Context, string, string, ...RequestOption) (*http.Request, error)
}

HttpClient is an interface that describes the methods of an http client.

The interface is intended to be used as a wrapper around an http.Client or other http client implementation, allowing for the addition of additional functionality or configuration.

func NewClient

func NewClient(name string, opts ...ClientOption) (HttpClient, error)

NewClient returns a new HttpClient with the name and url specified, wrapping a supplied ClientInterface implementation. Additional configuration options may be optionally specified.

params

name  // identifies the client, e.g. in errors
opts  // optional configuration

The url typically includes the protocol, hostname and port for the client but may include any additional url components consistently required for requests performed using the client.

type MockClient

type MockClient interface {
	Expect(method string, path string) *MockRequest
	ExpectDelete(path string) *MockRequest
	ExpectGet(path string) *MockRequest
	ExpectPatch(path string) *MockRequest
	ExpectPost(path string) *MockRequest
	ExpectPut(path string) *MockRequest
	ExpectationsWereMet() error
	Reset()
}

MockClient is an interface that described the methods provided for mocking expectations on a client

type MockExpectationsError

type MockExpectationsError struct {
	// contains filtered or unexported fields
}

MockExpectationsError is the error returned by ExpectationsNotMet() when one or more configured expectations have not been met. It wraps all errors representing the failed expectations.

func (MockExpectationsError) Error

func (err MockExpectationsError) Error() string

Error implements the error interface for MockExpectationsError by returning a string representation of the error, presenting each wrapped error indented under a summary identifying the mock client to which the failures relate.

type MockRequest

type MockRequest struct {

	// configuration of the response to be mocked in response to the request
	Response *mockResponse
	// contains filtered or unexported fields
}

MockRequest holds details of a request expected by a MockClient

func (MockRequest) String

func (rq MockRequest) String() string

String implements the stringer interface for a MockRequest, returning a string consisting of the request method (or <ANY> if not specified) and url (or <any://hostname/and/path> if not specified)

func (*MockRequest) WillNotBeCalled

func (mock *MockRequest) WillNotBeCalled()

WillNotBeCalled indicates that the request is not expected to be made. If a corresponding request is made by the client, this will be reflected as a failed expectation.

func (*MockRequest) WillRespond

func (mock *MockRequest) WillRespond() *mockResponse

WillRespond establishes a default response for the request, returning a mock response to be used to provide details of the response such as status code, headers or a body etc.

func (*MockRequest) WillReturnError

func (mock *MockRequest) WillReturnError(err error)

WillReturnError establishes an error to be returned by the client when attempting to perform this request. Any other response configuration is discarded if a request is configured to return an error.

func (*MockRequest) WithBody

func (mock *MockRequest) WithBody(b []byte) *MockRequest

WithBody identifies the expected body to be sent with the request.

func (*MockRequest) WithHeader

func (mock *MockRequest) WithHeader(k string, v ...string) *MockRequest

WithHeader identifies a header expected to be included with the request. The key (k) is normalised using textproto.CanonicalMIMEHeaderKey. An option value (v) may be specified; if no value is specified then the header only needs to be present; if a value is also specified then the header must be present with the specified value.

If multiple values are specified only the first is significant; additional values are discarded.

To configured a non-canonical header, use WithNonCanonicalHeader().

func (*MockRequest) WithNonCanonicalHeader

func (mock *MockRequest) WithNonCanonicalHeader(k string, v ...string) *MockRequest

WithNonCanonicalHeader identifies a non-canonical header expected to be included with the request. The key (k) is expected to match the case as specified. An option value (v) may be specified; if no value is specified then the header only needs to be present; if a value is also specified then the header must be present with the specified value.

If multiple values are specified only the first is significant; additional values are discarded.

To configured a canonical header, ensuring that the header key is normalised using textproto.CanonicalMIMEHeaderKey, use WithHeader().

type Request

type Request = http.Request

type RequestOption

type RequestOption = func(*http.Request) error

RequestOption is a function that applies an option to a request

type Response

type Response = http.Response

type ResponseWriter

type ResponseWriter = http.ResponseWriter

type RoundTripper

type RoundTripper = http.RoundTripper

type Transport

type Transport = http.Transport