limiter

Documentation

Index

• type Limiter
  • func NewLimiter(config RateLimitConfig, pool *radix.Pool) (Limiter, error)
  • func (l *Limiter) CanMakeRequest(requestWeight int) (bool, int64)
  • func (l *Limiter) GetStatus() RequestsStatus
  • func (l *Limiter) HitRateLimit(requestWeight int) error
  • func (l *Limiter) RequestCancelled(requestWeight int) error
  • func (l *Limiter) RequestSuccessful(requestWeight int) error
  • func (l *Limiter) WaitForRatelimit(requestWeight int)
• type RateLimitConfig
  • func NewRateLimitConfig(host string, sustainedRequestLimit int, sustainedTimePeriod int64, ...) RateLimitConfig
• type RequestsStatus

Examples

• Limiter.WaitForRatelimit

Types

type Limiter

type Limiter struct {
	// contains filtered or unexported fields
}

Limiter controls how often requests can be made. It uses a radix pool to connect to your redis database and the web api's RateLimitConfig to keep the number of allowed requests under the ratelimit.

The request weight of any request is how much it counts against the ratelimit. If an api's ratelimit allows 10 requests per second and a specific type of request counts for two of the 10 allowed requests per second, the request weight is two. However, in most cases the request weight is one.

func NewLimiter

func NewLimiter(config RateLimitConfig, pool *radix.Pool) (Limiter, error)

NewLimiter returns a new Limiter and requires a radix pool to allow the limiter to connect to the redis database. It also requires a RateLimitConfig so it can throttle requests to stay under the ratelimit while allowing as many requests as possible.

func (*Limiter) CanMakeRequest

func (l *Limiter) CanMakeRequest(requestWeight int) (bool, int64)

CanMakeRequest communicates with the database to figure out when it is possible to make a request. If a request can be made it returns true, 0. If a request cannot be made it returns false and the amount of time to sleep before your program should call CanMakeRequest again

func (*Limiter) GetStatus

func (l *Limiter) GetStatus() RequestsStatus

GetStatus returns the status of the requests, which includes the number on requests made in the period, the number of pending requests, the timestamp of the beginning of the period, and the timestamp for when the last error occurred.

func (*Limiter) HitRateLimit

func (l *Limiter) HitRateLimit(requestWeight int) error

HitRateLimit must be called only after CanMakeRequest returned true and a request has been completed with a status code of 429 or 419. This will automatically adjust the RateLimitConfig in the Limiter struct to prevent more 429s in the future.

func (*Limiter) RequestCancelled

func (l *Limiter) RequestCancelled(requestWeight int) error

RequestCancelled must be called if CanMakeRequest returned true, but the request to the api was never actually made.

func (*Limiter) RequestSuccessful

func (l *Limiter) RequestSuccessful(requestWeight int) error

RequestSuccessful must be called only after CanMakeRequest returned true and when a request has been completed and returned without a 429 or 419 status code

func (*Limiter) WaitForRatelimit

func (l *Limiter) WaitForRatelimit(requestWeight int)

WaitForRatelimit recursively calls CanMakeRequest until a request can be made. It handles the sleeping when a request cannot be made and it blocks until a request can be made.

Example

example function for godoc purposes

config := NewRateLimitConfig("testhost", 60, 60, 1, 1, 3)
limiter, err := NewLimiter(config, pool)
if err != nil {
	//handle err
}

for {
	limiter.WaitForRatelimit(1)

	//make api request
	statusCode, err := getStatusCode("www.example.com", 1)
	if err != nil {
		if err := limiter.RequestCancelled(1); err != nil {
			//handle error
		}

	}

	if statusCode == 429 {
		if err := limiter.HitRateLimit(1); err != nil {
			//handle error
		}
	} else {
		if err := limiter.RequestSuccessful(1); err != nil {
			//handle error
		}
	}
}

type RateLimitConfig

type RateLimitConfig struct {
	// contains filtered or unexported fields
}

RateLimitConfig struct contains the rate limit information for a specific api.

If you want to coordinate requests to one api across multiple threads, routines, containers, etc, it is imperative that each Limiter you create is initialized with a RateLimitConfig that has the same host name otherwise the Limiter structs will not be able to communicate and you will definitely hit the ratelimit.

func NewRateLimitConfig

func NewRateLimitConfig(host string, sustainedRequestLimit int, sustainedTimePeriod int64, burstRequestLimit int, burstTimePeriod int64, waitAfterHitLimit int64) RateLimitConfig

NewRateLimitConfig creates a rate limit config for a Limiter struct.

If you want to coordinate requests to one api across multiple threads, routines, containers, etc, it is imperative that each Limiter you create is initialized with a RateLimitConfig that has the same host name otherwise the Limiter structs will not be able to communicate and you will definitely hit the ratelimit

NewRateLimitConfig takes in two rates: a sustained ratelimit and a burst ratelimit. If the api you are making requests to only uses one ratelimit, enter in that rate for both the sustained and burst ratelimit.

config := NewRateLimitConfig("exampleHostName", 1200, 60, 20, 1)

The time periods of both rates are in terms of seconds so the config above has a sustained ratelimit of 1200 requests per 60 seconds and a burst ratelimit of 20 requests per second.

waitAfterHitLimit is the amount of time in milliseconds the limiter will wait before allowing more requests after hitting the ratelimit.

type RequestsStatus

type RequestsStatus struct {
	// contains filtered or unexported fields
}

RequestsStatus struct contains all info pertaining to the cumulative requests made to a specific host