httpclient

Documentation

Index

• Variables
• func SafeIP(ip netip.Addr) bool
• func ValidateRequest(r *http.Request) error
• type Client
  • func New(cfg Config) *Client
  • func (c *Client) Do(r *http.Request) (rsp *http.Response, err error)
  • func (c *Client) DoOnce(r *Request) (rsp *http.Response, retry bool, err error)
• type Config
• type Request
  • func WrapRequest(r *http.Request) Request
  • func (r *Request) BackOff() time.Duration
• type Sanitizer
  • func (s *Sanitizer) Sanitize(ntwrk, addr string, _ syscall.RawConn) error
• type SignFunc

Variables

var (
	// ErrInvalidRequest is returned if a given HTTP request is invalid and cannot be performed.
	ErrInvalidRequest = errors.New("invalid http request")

	// ErrInvalidNetwork is returned if the request would not be performed over TCP
	ErrInvalidNetwork = errors.New("invalid network type")

	// ErrReservedAddr is returned if a dialed address resolves to an IP within a blocked or reserved net.
	ErrReservedAddr = errors.New("dial within blocked / reserved IP range")

	// ErrBodyTooLarge is returned when a received response body is above predefined limit (default 40MB).
	ErrBodyTooLarge = errors.New("body size too large")
)

Functions

func SafeIP

func SafeIP(ip netip.Addr) bool

SafeIP returns whether ip is an IPv4/6 address in a non-reserved, public range.

func ValidateRequest

func ValidateRequest(r *http.Request) error

ValidateRequest performs the same request validation logic found in the default net/http.Transport{}.roundTrip() function, but pulls it out into this separate function allowing validation errors to be wrapped under a single error type.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client wraps an underlying http.Client{} to provide the following:

• setting a maximum received request body size, returning error on large content lengths, and using a limited reader in all other cases to protect against forged / unknown content-lengths
• protection from server side request forgery (SSRF) by only dialing out to known public IP prefixes, configurable with allows/blocks
• retry-backoff logic for error temporary HTTP error responses
• optional request signing
• request logging

func New

func New(cfg Config) *Client

New returns a new instance of Client initialized using configuration.

func (*Client) Do

func (c *Client) Do(r *http.Request) (rsp *http.Response, err error)

Do will essentially perform http.Client{}.Do() with retry-backoff functionality.

func (*Client) DoOnce

func (c *Client) DoOnce(r *Request) (rsp *http.Response, retry bool, err error)

DoOnce wraps an underlying http.Client{}.Do() to perform our wrapped request type: rewinding response body to permit reuse, signing request data when SignFunc provided, marking erroring hosts, updating retry attempt counts and setting backoff from header.

type Config

type Config struct {

	// MaxOpenConnsPerHost limits the max
	// number of open connections to a host.
	MaxOpenConnsPerHost int

	// AllowRanges allows outgoing
	// communications to given IP nets.
	AllowRanges []netip.Prefix

	// BlockRanges blocks outgoing
	// communiciations to given IP nets.
	BlockRanges []netip.Prefix

	// TLSInsecureSkipVerify can be set to true to
	// skip validation of remote TLS certificates.
	//
	// THIS SHOULD BE USED FOR TESTING ONLY, IF YOU
	// TURN THIS ON WHILE RUNNING IN PRODUCTION YOU
	// ARE LEAVING YOUR SERVER WIDE OPEN TO ATTACKS!
	TLSInsecureSkipVerify bool

	// MaxIdleConns: see http.Transport{}.MaxIdleConns.
	MaxIdleConns int

	// ReadBufferSize: see http.Transport{}.ReadBufferSize.
	ReadBufferSize int

	// WriteBufferSize: see http.Transport{}.WriteBufferSize.
	WriteBufferSize int

	// MaxBodySize determines the maximum fetchable body size.
	MaxBodySize int64

	// Timeout: see http.Client{}.Timeout.
	Timeout time.Duration

	// DisableCompression: see http.Transport{}.DisableCompression.
	DisableCompression bool
}

Config provides configuration details for setting up a new instance of httpclient.Client{}. Within are a subset of the configuration values passed to initialized http.Transport{} and http.Client{}, along with httpclient.Client{} specific.

type Request

type Request struct {

	// log fields.
	log.Entry

	// underlying request.
	*http.Request
	// contains filtered or unexported fields
}

Request wraps an HTTP request to add our own retry / backoff.

func WrapRequest

func WrapRequest(r *http.Request) Request

WrapRequest wraps an existing http.Request within our own httpclient.Request with retry / backoff tracking.

func (*Request) BackOff

func (r *Request) BackOff() time.Duration

GetBackOff returns the currently set backoff duration, (using a default according to no. attempts if needed).

type Sanitizer

type Sanitizer struct {
	Allow []netip.Prefix
	Block []netip.Prefix
}

func (*Sanitizer) Sanitize

func (s *Sanitizer) Sanitize(ntwrk, addr string, _ syscall.RawConn) error

Sanitize implements the required net.Dialer.Control function signature.

type SignFunc

type SignFunc func(r *http.Request) error

SignFunc is a function signature that provides request signing.