httpretry

README

A re-tryable HTTP client for Go

Safely re-try failing requests in Go with backoff.

Usage

A simple usage with some sensible defaults

client := hr.NewClient()

You can configure the library:

client.RetryMax = 100
client.RetryWaitMax = 60 * time.Second

You can also specify your own retry policy. By default only 503 errors are retried as per the RFCs:

// Retry on 500 errors, not 503, but also retry on 401
client.CheckForRetry = func(resp *http.Response, err error) (bool, error) {
	if err != nil {
		return true, err
	}
	if resp.StatusCode == 0 || resp.StatusCode >= 500 || resp.StatusCode == 401 {
		return true, nil
	}

	return false, nil
}

Calling underlying http.Client methods

The library wraps the standard HTTP client so you can call the underlying methods.

Adding a cookie jar

client := hr.NewClient()

cookieJar, err := cookiejar.New(nil)
if err != nil {
	panic(err)
}
client.HTTPClient.Jar = cookieJar

Setting headers and the content length

req, err := httpretry.NewRequestWithContext(ctx, "POST", url, data)
if err != nil {
	return nil, err
}
req.Header.Add("Content-Type", "application/json; charset=utf-8")
req.Header.Add("Accept", "application/json")
req.ContentLength = int64(len(data))

TODO

Handle context better in the client.Do retry loop

Ackbowledgements

This repo was based on, and inspired by this code, with a few changes and fixes.

Documentation

Index

• func DefaultBackoff(min, max time.Duration, attemptNum int, resp *http.Response) time.Duration
• func DefaultRetryPolicy(resp *http.Response, err error) (bool, error)
• type Backoff
• type CheckForRetry
• type Client
  • func NewClient() *Client
  • func (c *Client) Do(req *Request) (*http.Response, error)
  • func (c *Client) Get(url string) (*http.Response, error)
  • func (c *Client) Post(url, bodyType string, body io.ReadSeeker) (*http.Response, error)
• type Request
  • func NewRequestWithContext(ctx context.Context, method, url string, body io.ReadSeeker) (*Request, error)

Functions

func DefaultBackoff

func DefaultBackoff(min, max time.Duration, attemptNum int, resp *http.Response) time.Duration

DefaultBackoff provides a default callback for Client.Backoff which will perform exponential backoff based on the attempt number and limited by the provided minimum and maximum durations.

func DefaultRetryPolicy

func DefaultRetryPolicy(resp *http.Response, err error) (bool, error)

DefaultRetryPolicy provides a default callback for Client.CheckForRetry, which will retry on connection errors and server errors.

Types

type Backoff

type Backoff func(min, max time.Duration, attemptNum int, resp *http.Response) time.Duration

Backoff specifies a policy for how long to wait between retries. It is called after a failing request to determine the amount of time that should pass before trying again.

type CheckForRetry

type CheckForRetry func(resp *http.Response, err error) (bool, error)

CheckForRetry specifies a policy for handling retries. It is called following each request with the reponse and error values returned by the http.Client. If it returns false, the Client stops retrying and returns the response to the caller. If it returns an error, that error value is return in lieu of the error from the request.

type Client

type Client struct {
	HTTPClient   *http.Client  // Internal HTTP client
	RetryWaitMin time.Duration // Minimum time to wait
	RetryWaitMax time.Duration // Maximum time to wait
	RetryMax     int           // Maximum number of retries

	// CheckForRetry specifies the policy for handling retries, and is called
	// after each request.
	CheckForRetry CheckForRetry

	// Backoff specifies the policy for how long to wait between retries
	Backoff Backoff
}

Client is used to make TTP requests. It adds additional functionality like automatic retries to tolerate minor outages.

func NewClient

func NewClient() *Client

NewClient creates a new client with default settings

func (*Client) Do

func (c *Client) Do(req *Request) (*http.Response, error)

Do wraps calling an HTTP method with retries

func (*Client) Get

func (c *Client) Get(url string) (*http.Response, error)

Get is a convenience helper for doing simple GET requests

func (*Client) Post

func (c *Client) Post(url, bodyType string, body io.ReadSeeker) (*http.Response, error)

Post is a convenience helper for doing simple POST requests

type Request

type Request struct {

	// Embed an HTTP request directly. This makes a *Request act exacty
	// like an *http.Request so that all meta methods are supported.
	*http.Request
	// contains filtered or unexported fields
}

Request wraps the metadata needed to create HTTP requests

func NewRequestWithContext

func NewRequestWithContext(ctx context.Context, method, url string, body io.ReadSeeker) (*Request, error)

NewRequestWithContext creates a new wrapped request