retry

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 30, 2024 License: Apache-2.0 Imports: 8 Imported by: 211

README

Retry

GoDoc

Retry is a Go library for facilitating retry logic and backoff. It's highly extensible with full control over how and when retries occur. You can also write your own custom backoff functions by implementing the Backoff interface.

Features

  • Extensible - Inspired by Go's built-in HTTP package, this Go backoff and retry library is extensible via middleware. You can write custom backoff functions or use a provided filter.

  • Independent - No external dependencies besides the Go standard library, meaning it won't bloat your project.

  • Concurrent - Unless otherwise specified, everything is safe for concurrent use.

  • Context-aware - Use native Go contexts to control cancellation.

Usage

Here is an example use for connecting to a database using Go's database/sql package:

package main

import (
  "context"
  "database/sql"
  "log"
  "time"

  "github.com/sethvargo/go-retry"
)

func main() {
  db, err := sql.Open("mysql", "...")
  if err != nil {
    log.Fatal(err)
  }

  ctx := context.Background()
  if err := retry.Fibonacci(ctx, 1*time.Second, func(ctx context.Context) error {
    if err := db.PingContext(ctx); err != nil {
      // This marks the error as retryable
      return retry.RetryableError(err)
    }
    return nil
  }); err != nil {
    log.Fatal(err)
  }
}

Backoffs

In addition to your own custom algorithms, there are built-in algorithms for backoff in the library.

Constant

A very rudimentary backoff, just returns a constant value. Here is an example:

1s -> 1s -> 1s -> 1s -> 1s -> 1s

Usage:

NewConstant(1 * time.Second)
Exponential

Arguably the most common backoff, the next value is double the previous value. Here is an example:

1s -> 2s -> 4s -> 8s -> 16s -> 32s -> 64s

Usage:

NewExponential(1 * time.Second)
Fibonacci

The Fibonacci backoff uses the Fibonacci sequence to calculate the backoff. The next value is the sum of the current value and the previous value. This means retires happen quickly at first, but then gradually take slower, ideal for network-type issues. Here is an example:

1s -> 1s -> 2s -> 3s -> 5s -> 8s -> 13s

Usage:

NewFibonacci(1 * time.Second)

Modifiers (Middleware)

The built-in backoff algorithms never terminate and have no caps or limits - you control their behavior with middleware. There's built-in middleware, but you can also write custom middleware.

Jitter

To reduce the changes of a thundering herd, add random jitter to the returned value.

b := NewFibonacci(1 * time.Second)

// Return the next value, +/- 500ms
b = WithJitter(500*time.Millisecond, b)

// Return the next value, +/- 5% of the result
b = WithJitterPercent(5, b)
MaxRetries

To terminate a retry, specify the maximum number of retries. Note this is retries, not attempts. Attempts is retries + 1.

b := NewFibonacci(1 * time.Second)

// Stop after 4 retries, when the 5th attempt has failed. In this example, the worst case elapsed
// time would be 1s + 1s + 2s + 3s = 7s.
b = WithMaxRetries(4, b)
CappedDuration

To ensure an individual calculated duration never exceeds a value, use a cap:

b := NewFibonacci(1 * time.Second)

// Ensure the maximum value is 2s. In this example, the sleep values would be
// 1s, 1s, 2s, 2s, 2s, 2s...
b = WithCappedDuration(2 * time.Second, b)
WithMaxDuration

For a best-effort limit on the total execution time, specify a max duration:

b := NewFibonacci(1 * time.Second)

// Ensure the maximum total retry time is 5s.
b = WithMaxDuration(5 * time.Second, b)

Benchmarks

Here are benchmarks against some other popular Go backoff and retry libraries. You can run these benchmarks yourself via the benchmark/ folder. Commas and spacing fixed for clarity.

Benchmark/cenkalti-7      13,052,668     87.3 ns/op
Benchmark/lestrrat-7         902,044    1,355 ns/op
Benchmark/sethvargo-7    203,914,245     5.73 ns/op

Notes and Caveats

  • Randomization uses math/rand seeded with the Unix timestamp instead of crypto/rand.
  • Ordering of addition of multiple modifiers will make a difference. For example; ensure you add CappedDuration before WithMaxDuration, otherwise it may early out too early. Another example is you could add Jitter before or after capping depending on your desired outcome.

Documentation

Overview

Package retry provides helpers for retrying.

This package defines flexible interfaces for retrying Go functions that may be flakey or eventually consistent. It abstracts the "backoff" (how long to wait between tries) and "retry" (execute the function again) mechanisms for maximum flexibility. Furthermore, everything is an interface, so you can define your own implementations.

The package is modeled after Go's built-in HTTP package, making it easy to customize the built-in backoff with your own custom logic. Additionally, callers specify which errors are retryable by wrapping them. This is helpful with complex operations where only certain results should retry.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Constant 

func Constant(ctx context.Context, t time.Duration, f RetryFunc) error

Constant is a wrapper around Retry that uses a constant backoff. It panics if the given base is less than zero.

func Do 

func Do(ctx context.Context, b Backoff, f RetryFunc) error

Do wraps a function with a backoff to retry. The provided context is the same context passed to the RetryFunc.

Example (CustomRetry) 
ctx := context.Background()

b := retry.NewFibonacci(1 * time.Nanosecond)

// This example demonstrates selectively retrying specific errors. Only errors
// wrapped with RetryableError are eligible to be retried.
if err := retry.Do(ctx, retry.WithMaxRetries(3, b), func(ctx context.Context) error {
	resp, err := http.Get("https://google.com/")
	if err != nil {
		return err
	}
	defer resp.Body.Close()

	switch resp.StatusCode / 100 {
	case 4:
		return fmt.Errorf("bad response: %v", resp.StatusCode)
	case 5:
		return retry.RetryableError(fmt.Errorf("bad response: %v", resp.StatusCode))
	default:
		return nil
	}
}); err != nil {
	// handle error
}

Output:
Example (Simple) 
ctx := context.Background()

b := retry.NewFibonacci(1 * time.Nanosecond)

i := 0
if err := retry.Do(ctx, retry.WithMaxRetries(3, b), func(ctx context.Context) error {
	fmt.Printf("%d\n", i)
	i++
	return retry.RetryableError(fmt.Errorf("oops"))
}); err != nil {
	// handle error
}

Output:

0
1
2
3

func DoValue added in v0.3.0 

func DoValue[T any](ctx context.Context, b Backoff, f RetryFuncValue[T]) (T, error)
Example 
ctx := context.Background()

b := retry.NewFibonacci(1 * time.Nanosecond)

body, err := retry.DoValue(ctx, retry.WithMaxRetries(3, b), func(ctx context.Context) ([]byte, error) {
	resp, err := http.Get("https://google.com/")
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()

	switch resp.StatusCode / 100 {
	case 4:
		return nil, fmt.Errorf("bad response: %v", resp.StatusCode)
	case 5:
		return nil, retry.RetryableError(fmt.Errorf("bad response: %v", resp.StatusCode))
	default:
		b, _ := io.ReadAll(resp.Body)
		return b, nil
	}
})
if err != nil {
	// handle error
}
_ = body

Output:

func Exponential 

func Exponential(ctx context.Context, base time.Duration, f RetryFunc) error

Exponential is a wrapper around Retry that uses an exponential backoff. See NewExponential.

func Fibonacci 

func Fibonacci(ctx context.Context, base time.Duration, f RetryFunc) error

Fibonacci is a wrapper around Retry that uses a Fibonacci backoff. See NewFibonacci.

func RetryableError 

func RetryableError(err error) error

RetryableError marks an error as retryable.

Types

type Backoff 

type Backoff interface {
	// Next returns the time duration to wait and whether to stop.
	Next() (next time.Duration, stop bool)
}

Backoff is an interface that backs off.

func NewConstant 

func NewConstant(t time.Duration) Backoff

NewConstant creates a new constant backoff using the value t. The wait time is the provided constant value. It panics if the given base is less than zero.

Example 
b := retry.NewConstant(1 * time.Second)

for i := 0; i < 5; i++ {
	val, _ := b.Next()
	fmt.Printf("%v\n", val)
}

Output:

1s
1s
1s
1s
1s

func NewExponential 

func NewExponential(base time.Duration) Backoff

NewExponential creates a new exponential backoff using the starting value of base and doubling on each failure (1, 2, 4, 8, 16, 32, 64...), up to max.

Once it overflows, the function constantly returns the maximum time.Duration for a 64-bit integer.

It panics if the given base is less than zero.

Example 
b := retry.NewExponential(1 * time.Second)

for i := 0; i < 5; i++ {
	val, _ := b.Next()
	fmt.Printf("%v\n", val)
}

Output:

1s
2s
4s
8s
16s

func NewFibonacci 

func NewFibonacci(base time.Duration) Backoff

NewFibonacci creates a new Fibonacci backoff using the starting value of base. The wait time is the sum of the previous two wait times on each failed attempt (1, 1, 2, 3, 5, 8, 13...).

Once it overflows, the function constantly returns the maximum time.Duration for a 64-bit integer.

It panics if the given base is less than zero.

Example 
b := retry.NewFibonacci(1 * time.Second)

for i := 0; i < 5; i++ {
	val, _ := b.Next()
	fmt.Printf("%v\n", val)
}

Output:

1s
2s
3s
5s
8s

func WithCappedDuration 

func WithCappedDuration(cap time.Duration, next Backoff) Backoff

WithCappedDuration sets a maximum on the duration returned from the next backoff. This is NOT a total backoff time, but rather a cap on the maximum value a backoff can return. Without another middleware, the backoff will continue infinitely.

Example 
ctx := context.Background()

b := retry.NewFibonacci(1 * time.Second)
b = retry.WithCappedDuration(3*time.Second, b)

if err := retry.Do(ctx, b, func(_ context.Context) error {
	// TODO: logic here
	return nil
}); err != nil {
	// handle error
}

Output:

func WithJitter 

func WithJitter(j time.Duration, next Backoff) Backoff

WithJitter wraps a backoff function and adds the specified jitter. j can be interpreted as "+/- j". For example, if j were 5 seconds and the backoff returned 20s, the value could be between 15 and 25 seconds. The value can never be less than 0.

Example 
ctx := context.Background()

b := retry.NewFibonacci(1 * time.Second)
b = retry.WithJitter(1*time.Second, b)

if err := retry.Do(ctx, b, func(_ context.Context) error {
	// TODO: logic here
	return nil
}); err != nil {
	// handle error
}

Output:

func WithJitterPercent 

func WithJitterPercent(j uint64, next Backoff) Backoff

WithJitterPercent wraps a backoff function and adds the specified jitter percentage. j can be interpreted as "+/- j%". For example, if j were 5 and the backoff returned 20s, the value could be between 19 and 21 seconds. The value can never be less than 0 or greater than 100.

Example 
ctx := context.Background()

b := retry.NewFibonacci(1 * time.Second)
b = retry.WithJitterPercent(5, b)

if err := retry.Do(ctx, b, func(_ context.Context) error {
	// TODO: logic here
	return nil
}); err != nil {
	// handle error
}

Output:

func WithMaxDuration 

func WithMaxDuration(timeout time.Duration, next Backoff) Backoff

WithMaxDuration sets a maximum on the total amount of time a backoff should execute. It's best-effort, and should not be used to guarantee an exact amount of time.

Example 
ctx := context.Background()

b := retry.NewFibonacci(1 * time.Second)
b = retry.WithMaxDuration(5*time.Second, b)

if err := retry.Do(ctx, b, func(_ context.Context) error {
	// TODO: logic here
	return nil
}); err != nil {
	// handle error
}

Output:

func WithMaxRetries 

func WithMaxRetries(max uint64, next Backoff) Backoff

WithMaxRetries executes the backoff function up until the maximum attempts.

Example 
ctx := context.Background()

b := retry.NewFibonacci(1 * time.Second)
b = retry.WithMaxRetries(3, b)

if err := retry.Do(ctx, b, func(_ context.Context) error {
	// TODO: logic here
	return nil
}); err != nil {
	// handle error
}

Output:

type BackoffFunc 

type BackoffFunc func() (time.Duration, bool)

BackoffFunc is a backoff expressed as a function.

Example 
ctx := context.Background()

// Example backoff middleware that adds the provided duration t to the result.
withShift := func(t time.Duration, next retry.Backoff) retry.BackoffFunc {
	return func() (time.Duration, bool) {
		val, stop := next.Next()
		if stop {
			return 0, true
		}
		return val + t, false
	}
}

// Middlewrap wrap another backoff:
b := retry.NewFibonacci(1 * time.Second)
b = withShift(5*time.Second, b)

if err := retry.Do(ctx, b, func(ctx context.Context) error {
	// Actual retry logic here
	return nil
}); err != nil {
	// handle error
}

Output:

func (BackoffFunc) Next 

func (b BackoffFunc) Next() (time.Duration, bool)

Next implements Backoff.

type RetryFunc 

type RetryFunc func(ctx context.Context) error

RetryFunc is a function passed to Do.

type RetryFuncValue added in v0.3.0 

type RetryFuncValue[T any] func(ctx context.Context) (T, error)

RetryFuncValue is a function passed to Do which returns a value.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.