retry

Documentation

Overview

Package retry provides utilities to retry an operation multiple times.

Index

• Constants
• Variables
• type ExpBackOff
  • func (ebo ExpBackOff) Retry(retries int, f func() error) error
  • func (ebo ExpBackOff) RetrySome(retries int, f func() (recoverable bool, err error)) error
  • func (ebo ExpBackOff) RetryWithCtx(ctx context.Context, retries int, f Func) error
• type FError
  • func (e *FError) Error() string
  • func (e *FError) Is(target error) bool
  • func (e *FError) Unwrap() error
• type Func

Constants

const Forever = -1

Forever can be used as a number of retries to retry forever. If you retry forever, the number of errors can grow without bound.

Variables

var (
	// Quick expects that failure conditions resolve quickly.
	// The first 10 attempts accumulate about a minute of total wait time,
	// after which each attempt occurs ~30 seconds after the previous.
	Quick = ExpBackOff{
		BackOff:  50 * time.Millisecond,
		Max:      30 * time.Second,
		Jitter:   true,
		KeepErrs: 10,
	}

	// Slow expects that failure conditions may take awhile to resolve.
	// The first 10 attempts accumulate about an hour of total wait time,
	// after which each subsequent event occurs ~30 minutes after the previous.
	Slow = ExpBackOff{
		BackOff:  5 * time.Second,
		Max:      30 * time.Minute,
		Jitter:   true,
		KeepErrs: 10,
	}

	// ErrRetriesExceeded is returned as the MainErr of an FError
	// when a Retry operation fails more times than permitted.
	ErrRetriesExceeded = errors.New("retries exceeded")

	// ErrWaitExceedsDeadline is returned if the wait time before the next attempt
	// exceeds a Deadline present on a context.Context associated with the operation.
	ErrWaitExceedsDeadline = errors.New("waiting would exceed the Deadline")
)

Types

type ExpBackOff

type ExpBackOff struct {
	BackOff  time.Duration
	Max      time.Duration
	KeepErrs int
	Jitter   bool
}

ExpBackOff is used to call a function multiple times, waiting an exponentially increasing amount of time between attempts.

ExpBackOff holds parameters for use in its Retry method. Between attempts, Retry waits multiple of the BackOff duration, where the multiple increases exponentially with the number of attempts. The delay between attempts will never exceed Max.

If Jitter is false, the multiple is simply increasing powers of 2. If Jitter is true, the multiple is a pseudo-random integer selected with equal probability from an interval twice the non-Jitter size, so that the expected value of both is the same.

Concretely, if BackOff is 1s, the expected delay between attempt n and n+1 is 1s, 2s, 4s, 16s, etc., and the total wait up to attempt n is 1s, 3s, 7s, 15s, etc. With Jitter, these are the exact wait times (up to the OS-precision, of course), whereas with Jitter, these are the mean wait times. Changing the BackOff duration scales these linearly: a BackOff of 3s yields expected waits of 3s, 6s, 12s, etc.

If KeepErrs is <=0, a failed Retry returns an *FError with only the MainError. If it's >0, it limits the maximum number of Other errors *FError records.

Since the zero value has a Max duration of 0, it retries the function as fast as possible. The global package variables can be used for common Retry behavior, otherwise create a new ExpBackOff with values appropriate for your use.

func (ExpBackOff) Retry

func (ebo ExpBackOff) Retry(retries int, f func() error) error

Retry attempts f up to retries number of times until it returns nil.

If the program receives SIGINT or SIGKILL, the retries are canceled; use RetryWithCtx if you wish to control this behavior.

If Retry returns non-nil, it is of type *FError.

func (ExpBackOff) RetrySome

func (ebo ExpBackOff) RetrySome(retries int, f func() (recoverable bool, err error)) error

RetrySome retries f until one of the following is true: - f returns a nil error - f returns "false" for recoverable (indicating an unrecoverable error) - f has been called "retries" times - the program receives SIGINT or SIGKILL

This last condition can be controlled by using RetryWithCtx instead.

Errors received from f are wrapped in *FError.

func (ExpBackOff) RetryWithCtx

func (ebo ExpBackOff) RetryWithCtx(ctx context.Context, retries int, f Func) error

RetryWithCtx works like RetrySome, but allows a custom context, which may be used to cancel during waits between attempts. The context is passed to f unmodified.

Regardless of the return values of f, if the context is canceled, Retry will not continue calling f; however, Retry can only check the context between calls to f. It is up to f to determine how/if to use the context. For short lived functions the don't await any signals, it's fine to ignore the context. For functions that block waiting on, e.g., a network resource, they should add ctx.Done to their select statements.

In most cases, it's not really necessary for f to check ctx immediately. Before calls to f, Retry checks if ctx is canceled, past its Deadline, or if its Deadline would occur before the next call to f. In these cases, it adds ctx.Err to its accumulated errors and returns. As a result, at the beginning of f's execution, it's unlikely (though possible) that ctx is canceled or past its deadline.

type FError

type FError struct {
	MainErr  error
	Others   []error
	Attempts int
	// contains filtered or unexported fields
}

FError records errors accumulated during each execution of f. FError is only returned if every f() attempt fails or retries are canceled.

If a call to one of the Retry methods results in a successful call of f(), any errors accumulated during the operation are discarded. Since it cannot be known whether or not f() will eventually be successful, this structure provides some history of those errors along with some stdlib-compatible methods for reviewing those errors.

FError.MainErr records the main reason the Retry operation failed. This may be ErrRetriesExceeded, ErrWaitExceedsDeadline, the non-nil return value from calling Err() on a user-supplied context.Context, or a non-nil error returned by f() itself. Calls to FError.Unwrap return this value.

If len(FError.Others) > 0, each error it holds is non-nil; however, it may contain fewer errors than the maximum number of attempts, if retries are canceled or if the capacity is limited by the retry mechanism. In the latter case, they may be in a different order then f() attempts.

func (*FError) Error

func (e *FError) Error() string

Error returns a string describing the MainError that caused FError, followed by any saved errors encountered during each Retry attempt, separated by newlines and indented by one tab.

func (*FError) Is

func (e *FError) Is(target error) bool

Is tests whether FError matches a target.

If the target is not an *FError, this returns true if MainErr matches the target, or if _all_ the Other errors in FError match the target, as determined by errors.Is(e.Others[i], target).

If the target is an *FError, this compares their MainErr and Others errors. It returns true if errors.Is(e.MainErr, target.MainErr) is true, they have the same number of errors in Others, and errors.Is(e.Others[i], target.Others[i]) is true for all i.

func (*FError) Unwrap

func (e *FError) Unwrap() error

Unwrap returns the MainErr for this FError.

type Func

type Func func(ctx context.Context) (bool, error)

Func represents a function which can be retried. When it returns a non-nil error, it also returns a boolean which is true if the error is temporary (and the Func should be retried) or false if the error will never recover on its own.