backoff

README

go-backoff

Backoff algorithm and helpers for Go

SYNOPSIS

import "github.com/lestrrat-go/backoff"

func Func(arg Foo) (Result, error) { ... }

var policy = backoff.NewExponential(
  backoff.WithInterval(500*time.Millisecond), // base interval
  backoff.WithJitterFactor(0.05), // 5% jitter
  backoff.WithMaxRetries(25), // If not specified, default number of retries is 10
)
func RetryFunc(arg Foo) (Result, error) {
  b, cancel := policy.Start(context.Background())
  defer cancel()

  for backoff.Continue(b) {
    res, err := Func(arg)
    if err == nil {
      return res, nil
    }
  }
  return nil, errors.New(`tried very hard, but no luck`)
}

Simple Usage

func ExampleRetry() {
  count := 0
  e := backoff.ExecuteFunc(func(_ context.Context) error {
    // This is a silly example that succeeds on every 10th try
    count++
    if count%10 == 0 {
      return nil
    }
    return errors.New(`dummy`)
  })

  ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
  defer cancel()

  // Note that the following will only retry up to 10 attempts,
  // as it's the default. If you want to wait retry forever, use
  // backoff.WithRetryForever (or `backoff.WithMaxRetries(0)`)
  p := backoff.NewExponential()
  if err := backoff.Retry(ctx, p, e); err != nil {
    log.Printf("failed to call function after repeated tries")
  }
}

DESCRIPTION

This library is an implementation of backoff algorithm for retrying operations in an idiomatic Go way. It respects context.Context natively, and the critical notifications are done through channel operations, allowing you to write code that is both more explicit and flexibile.

It also exports a utility function Retry, for simple operations.

For a longer discussion, please read this article

CONSTRUCTOR OPTIONS

WithFactor

TODO

WithInterval

TODO

WithJitterFactor

TODO

WithMaxInterval

WithMaxInterval specifies the maximum interval between retries, and is currently only applicable to exponential backoffs.

By default this is capped at 2 minutes. If you would like to change this value, you must explicitly specify it through this option.

If a value of 0 is specified, then there is no limit, and the backoff interval will keep increasing.

WithMaxRetries

WithMaxRetries specifies the maximum number of attempts that can be made by the backoff policies. By default each policy tries up to 10 times.

If you would like to retry forever, specify "0" and pass to the constructor of each policy.

WithRetryForever

WithRetryForever is a short-hand for WithMaxRetries(0) -- well, ok, it's not a short-hand. But it makes reading the code just a tiny bit easier

WithMaxElapsedTime

WithMaxElapsedTime specifies the maximum amount of accumulative time that the backoff is allowed to wait before it is considered failed.

TODO

• Refine the timeouts generated by exponential backoff algorithm (help wanted!)

PRIOR ARTS

github.com/cenkalti/backoff

This library is featureful, but one thing that gets to me is the fact that it essentially forces you to create a closure over the operation to be retried.

github.com/jpillora/backoff

This library is a very simple implementation of calculating backoff durations. I wanted it to let me know when to stop too, so it was missing a few things.

DUMB BENCHMARK

make benchmark
go test -run=none -tags bench -bench . -benchmem -benchtime 20s
goos: darwin
goarch: amd64
pkg: github.com/lestrrat-go/backoff
Benchmark/cenkalti-4         	       3	6828914804 ns/op	    1205 B/op	      24 allocs/op
Benchmark/lestrrat-4         	       5	4842358461 ns/op	    1022 B/op	      20 allocs/op
PASS
ok  	github.com/lestrrat-go/backoff	70.711s

Documentation

Overview

Package backoff implments backoff algorithms for retrying operations.

Users first create an appropriate `Policy` object, and when the operation that needs retrying is about to start, they kick the actual backoff

Example

package main

import (
	"context"
	"errors"
	"time"

	backoff "github.com/lestrrat-go/backoff"
)

func main() {
	p := backoff.NewConstant(time.Second)

	flakyFunc := func(a int) (int, error) {
		// silly function that only succeeds if the current call count is
		// divisible by either 3 or 5 but not both
		switch {
		case a%15 == 0:
			return 0, errors.New(`invalid`)
		case a%3 == 0 || a%5 == 0:
			return a, nil
		}
		return 0, errors.New(`invalid`)
	}

	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	retryFunc := func(v int) (int, error) {
		b, cancel := p.Start(ctx)
		defer cancel()

		for {
			x, err := flakyFunc(v)
			if err == nil {
				return x, nil
			}

			select {
			case <-b.Done():
				return 0, errors.New(`....`)
			case <-b.Next():
				// no op, go to next
			}
		}
	}

	retryFunc(15)
}

Index

• func Continue(b Backoff) bool
• func IsPermanentError(err error) bool
• func MarkPermanent(err error) error
• func Retry(ctx context.Context, p Policy, e Executer) error
• type Backoff
• type CancelFunc
• type Constant
  • func NewConstant(delay time.Duration, options ...Option) *Constant
  • func (p *Constant) Start(ctx context.Context) (Backoff, CancelFunc)
• type ExecuteFunc
  • func (f ExecuteFunc) Execute(ctx context.Context) error
• type Executer
• type Exponential
  • func NewExponential(options ...Option) *Exponential
  • func (p *Exponential) Start(ctx context.Context) (Backoff, CancelFunc)
• type Option
  • func WithFactor(v float64) Option
  • func WithInterval(v time.Duration) Option
  • func WithJitterFactor(v float64) Option
  • func WithMaxElapsedTime(v time.Duration) Option
  • func WithMaxInterval(v time.Duration) Option
  • func WithMaxRetries(v int) Option
  • func WithRetryForever() Option
• type PermanentError
• type Policy

Examples

• Package
• Continue
• Retry

Functions

func Continue

func Continue(b Backoff) bool

Continue is a convenience wrapper around the

Example

package main

import (
	"context"
	"errors"
	"time"

	backoff "github.com/lestrrat-go/backoff"
)

func main() {
	p := backoff.NewConstant(time.Second)

	flakyFunc := func(a int) (int, error) {
		// silly function that only succeeds if the current call count is
		// divisible by either 3 or 5 but not both
		switch {
		case a%15 == 0:
			return 0, errors.New(`invalid`)
		case a%3 == 0 || a%5 == 0:
			return a, nil
		}
		return 0, errors.New(`invalid`)
	}

	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	retryFunc := func(v int) (int, error) {
		b, cancel := p.Start(ctx)
		defer cancel()

		for backoff.Continue(b) {
			x, err := flakyFunc(v)
			if err == nil {
				return x, nil
			}
		}
		return 0, errors.New(`failed to get value`)
	}

	retryFunc(15)
}

func IsPermanentError

func IsPermanentError(err error) bool

IsPermanentError returns true if the given error is a permanent error. Permanent errors are those that implements the `PermanentError` interface and returns `true` for the `IsPermanent` method.

func MarkPermanent

func MarkPermanent(err error) error

func Retry

func Retry(ctx context.Context, p Policy, e Executer) error

Retry is a convenience wrapper around the backoff algorithm. If your target operation can be nicely enclosed in the `Executer` interface, this will remove your need to write much of the boilerplate.

Example

package main

import (
	"context"
	"errors"
	"log"
	"time"

	backoff "github.com/lestrrat-go/backoff"
)

func main() {
	count := 0
	e := backoff.ExecuteFunc(func(_ context.Context) error {
		// This is a silly example that succeeds on every 10th try
		count++
		if count%10 == 0 {
			return nil
		}
		return errors.New(`dummy`)
	})

	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	p := backoff.NewExponential()
	if err := backoff.Retry(ctx, p, e); err != nil {
		log.Printf("failed to call function after repeated tries")
	}
}

Types

type Backoff

type Backoff interface {
	Done() <-chan struct{}
	Next() <-chan struct{}
}

type CancelFunc

type CancelFunc func()

type Constant

type Constant struct {
	// contains filtered or unexported fields
}

func NewConstant

func NewConstant(delay time.Duration, options ...Option) *Constant

func (*Constant) Start

func (p *Constant) Start(ctx context.Context) (Backoff, CancelFunc)

type ExecuteFunc

type ExecuteFunc func(context.Context) error

ExecuteFunc is an Executer that is represented by a single function

func (ExecuteFunc) Execute

func (f ExecuteFunc) Execute(ctx context.Context) error

Execute executes the operation

type Executer

type Executer interface {
	Execute(context.Context) error
}

Executer represents an operation that can be performed within the Retry utility method.

type Exponential

type Exponential struct {
	// contains filtered or unexported fields
}

Exponential implements an exponential backoff policy.

func NewExponential

func NewExponential(options ...Option) *Exponential

func (*Exponential) Start

func (p *Exponential) Start(ctx context.Context) (Backoff, CancelFunc)

type Option

type Option interface {
	Name() string
	Value() interface{}
}

func WithFactor

func WithFactor(v float64) Option

func WithInterval

func WithInterval(v time.Duration) Option

func WithJitterFactor

func WithJitterFactor(v float64) Option

func WithMaxElapsedTime

func WithMaxElapsedTime(v time.Duration) Option

WithMaxElapsedTime specifies the maximum amount of accumulative time that the backoff is allowed to wait before it is considered failed.

func WithMaxInterval

func WithMaxInterval(v time.Duration) Option

WithMaxInterval specifies the maximum interval between retries, and is currently only applicable to exponential backoffs.

By default this is capped at 2 minutes. If you would like to change this value, you must explicitly specify it through this option.

If a value of 0 is specified, then there is no limit, and the backoff interval will keep increasing.

func WithMaxRetries

func WithMaxRetries(v int) Option

WithMaxRetries specifies the maximum number of attempts that can be made by the backoff policies. By default each policy tries up to 10 times.

If you would like to retry forever, specify "0" and pass to the constructor of each policy.

func WithRetryForever

func WithRetryForever() Option

WithRetryForever is a short-hand for `WithMaxRetries(0)` -- well, ok, it's not a short-hand. But it makes reading the code just a tiny bit easier

type PermanentError

type PermanentError interface {
	IsPermanent() bool
}

type Policy

type Policy interface {
	Start(context.Context) (Backoff, CancelFunc)
}