limiter

README

Limiter

Dead simple rate limit middleware for Go.

• Simple API
• "Store" approach for backend
• Redis support (but not tied too)
• Middlewares: HTTP and Gin

Installation

$ go get github.com/ulule/limiter

Usage

In five steps:

• Create a limiter.Rate instance (the number of requests per period)
• Create a limiter.Store instance (see Redis or In-Memory)
• Create a limiter.Limiter instance that takes store and rate instances as arguments
• Create a middleware instance using the middleware of your choice
• Give the limiter instance to your middleware initializer

Example:

// Create a rate with the given limit (number of requests) for the given
// period (a time.Duration of your choice).
import "github.com/ulule/limiter"

rate := limiter.Rate{
    Period: 1 * time.Hour,
    Limit:  1000,
}

// You can also use the simplified format "<limit>-<period>"", with the given
// periods:
//
// * "S": second
// * "M": minute
// * "H": hour
//
// Examples:
//
// * 5 reqs/second: "5-S"
// * 10 reqs/minute: "10-M"
// * 1000 reqs/hour: "1000-H"
//
rate, err := limiter.NewRateFromFormatted("1000-H")
if err != nil {
    panic(err)
}

// Then, create a store. Here, we use the bundled Redis store. Any store
// compliant to limiter.Store interface will do the job. The defaults are
// "limiter" as Redis key prefix and a maximum of 3 retries for the key under
// race condition.
import "github.com/ulule/limiter/drivers/store/redis"

store, err := redis.NewStore(client)
if err != nil {
    panic(err)
}

// Alternatively, you can pass options to the store with the "WithOptions"
// function. For example, for Redis store:
import "github.com/ulule/limiter/drivers/store/redis"

store, err := redis.NewStoreWithOptions(pool, limiter.StoreOptions{
    Prefix:   "your_own_prefix",
    MaxRetry: 4,
})
if err != nil {
    panic(err)
}

// Or use a in-memory store with a goroutine which clears expired keys.
import "github.com/ulule/limiter/drivers/store/memory"

store := memory.NewStore()

// Then, create the limiter instance which takes the store and the rate as arguments.
// Now, you can give this instance to any supported middleware.
instance := limiter.New(store, rate)

See middleware examples:

• HTTP
• Gin

How it works

The ip address of the request is used as a key in the store.

If the key does not exist in the store we set a default value with an expiration period.

You will find two stores:

• Redis: rely on TTL and incrementing the rate limit on each request.
• In-Memory: rely on a fork of go-cache with a goroutine to clear expired keys using a default interval.

When the limit is reached, a 429 HTTP status code is sent.

Why Yet Another Package

You could ask us: why yet another rate limit package?

Because existing packages did not suit our needs.

We tried a lot of alternatives:

1. Throttled. This package uses the generic cell-rate algorithm. To cite the documentation: "The algorithm has been slightly modified from its usual form to support limiting with an additional quantity parameter, such as for limiting the number of bytes uploaded". It is brillant in term of algorithm but documentation is quite unclear at the moment, we don't need burst feature for now, impossible to get a correct After-Retry (when limit exceeds, we can still make a few requests, because of the max burst) and it only supports http.Handler middleware (we use Gin). Currently, we only need to return 429 and X-Ratelimit-* headers for n reqs/duration.

2. Speedbump. Good package but maybe too lightweight. No Reset support, only one middleware for Gin framework and too Redis-coupled. We rather prefer to use a "store" approach.

3. Tollbooth. Good one too but does both too much and too little. It limits by remote IP, path, methods, custom headers and basic auth usernames... but does not provide any Redis support (only in-memory) and a ready-to-go middleware that sets X-Ratelimit-* headers. tollbooth.LimitByRequest(limiter, r) only returns an HTTP code.

4. ratelimit. Probably the closer to our needs but, once again, too lightweight, no middleware available and not active (last commit was in August 2014). Some parts of code (Redis) comes from this project. It should deserve much more love.

There are other many packages on GitHub but most are either too lightweight, too old (only support old Go versions) or unmaintained. So that's why we decided to create yet another one.

Contributing

• Ping us on twitter:
  • @oibafsellig
  • @thoas
  • @novln_
• Fork the project
• Fix bugs

Don't hesitate ;)

Documentation

Index

• Constants
• func GetIP(r *http.Request, trustForwardHeader ...bool) net.IP
• func GetIPKey(r *http.Request, trustForwardHeader ...bool) string
• func Random(min, max int) int
• type Context
• type Limiter
  • func New(store Store, rate Rate) *Limiter
  • func (limiter *Limiter) Get(ctx context.Context, key string) (Context, error)
  • func (limiter *Limiter) Peek(ctx context.Context, key string) (Context, error)
• type Rate
  • func NewRateFromFormatted(formatted string) (Rate, error)
• type Store
• type StoreOptions

Constants

const (
	// DefaultPrefix is the default prefix to use for the key in the store.
	DefaultPrefix = "limiter"

	// DefaultMaxRetry is the default maximum number of key retries under
	// race condition (mainly used with database-based stores).
	DefaultMaxRetry = 3

	// DefaultCleanUpInterval is the default time duration for cleanup.
	DefaultCleanUpInterval = 30 * time.Second
)

Functions

func GetIP

func GetIP(r *http.Request, trustForwardHeader ...bool) net.IP

GetIP returns IP address from request.

func GetIPKey

func GetIPKey(r *http.Request, trustForwardHeader ...bool) string

GetIPKey extracts IP from request and returns hashed IP to use as store key.

func Random

func Random(min, max int) int

Random return a random integer between min and max.

Types

type Context

type Context struct {
	Limit     int64
	Remaining int64
	Reset     int64
	Reached   bool
}

Context is the limit context.

type Limiter

type Limiter struct {
	Store Store
	Rate  Rate
}

Limiter is the limiter instance.

func New

func New(store Store, rate Rate) *Limiter

New returns an instance of Limiter.

func (*Limiter) Get

func (limiter *Limiter) Get(ctx context.Context, key string) (Context, error)

Get returns the limit for given identifier.

func (*Limiter) Peek

func (limiter *Limiter) Peek(ctx context.Context, key string) (Context, error)

Peek returns the limit for given identifier, without modification on current values.

type Rate

type Rate struct {
	Formatted string
	Period    time.Duration
	Limit     int64
}

Rate is the rate.

func NewRateFromFormatted

func NewRateFromFormatted(formatted string) (Rate, error)

NewRateFromFormatted returns the rate from the formatted version.

type Store

type Store interface {
	// Get returns the limit for given identifier.
	Get(ctx context.Context, key string, rate Rate) (Context, error)
	// Peek returns the limit for given identifier, without modification on current values.
	Peek(ctx context.Context, key string, rate Rate) (Context, error)
}

Store is the common interface for limiter stores.

type StoreOptions

type StoreOptions struct {
	// Prefix is the prefix to use for the key.
	Prefix string

	// MaxRetry is the maximum number of retry under race conditions.
	MaxRetry int

	// CleanUpInterval is the interval for cleanup.
	CleanUpInterval time.Duration
}

StoreOptions are options for store.