github_ratelimit

Documentation

Index

• Constants
• func NewRateLimitWaiterClient(base http.RoundTripper, opts ...Option) (*http.Client, error)
• func WithOverrideConfig(ctx context.Context, opts ...Option) context.Context
• type CallbackContext
• type OnLimitDetected
• type OnSingleLimitExceeded
• type OnTotalLimitExceeded
• type Option
  • func GetConfigOverrides(ctx context.Context) []Option
  • func WithLimitDetectedCallback(callback OnLimitDetected) Option
  • func WithSingleSleepLimit(limit time.Duration, callback OnSingleLimitExceeded) Option
  • func WithTotalSleepLimit(limit time.Duration, callback OnTotalLimitExceeded) Option
• type SecondaryRateLimitBody
  • func (s SecondaryRateLimitBody) IsSecondaryRateLimit() bool
• type SecondaryRateLimitConfig
  • func (c *SecondaryRateLimitConfig) ApplyOptions(opts ...Option)
  • func (c *SecondaryRateLimitConfig) IsAboveSingleSleepLimit(sleepTime time.Duration) bool
  • func (c *SecondaryRateLimitConfig) IsAboveTotalSleepLimit(sleepTime time.Duration, totalSleepTime time.Duration) bool
• type SecondaryRateLimitWaiter
  • func NewRateLimitWaiter(base http.RoundTripper, opts ...Option) (*SecondaryRateLimitWaiter, error)
  • func (t *SecondaryRateLimitWaiter) RoundTrip(request *http.Request) (*http.Response, error)

Constants

const (
	HeaderRetryAfter          = "retry-after"
	HeaderXRateLimitReset     = "x-ratelimit-reset"
	HeaderXRateLimitRemaining = "x-ratelimit-remaining"
)

const (
	SecondaryRateLimitMessage                 = `You have exceeded a secondary rate limit`
	SecondaryRateLimitDocumentationPathSuffix = `secondary-rate-limits`
)

Functions

func NewRateLimitWaiterClient

func NewRateLimitWaiterClient(base http.RoundTripper, opts ...Option) (*http.Client, error)

func WithOverrideConfig

func WithOverrideConfig(ctx context.Context, opts ...Option) context.Context

WithOverrideConfig adds config overrides to the context. The overrides are applied on top of the existing config. Allows for request-specific overrides.

Types

type CallbackContext

type CallbackContext struct {
	RoundTripper   *SecondaryRateLimitWaiter
	SleepUntil     *time.Time
	TotalSleepTime *time.Duration
	Request        *http.Request
	Response       *http.Response
}

CallbackContext is passed to all callbacks. Fields might be nillable, depending on the specific callback and field.

type OnLimitDetected

type OnLimitDetected func(*CallbackContext)

OnLimitDetected is a callback to be called when a new rate limit is detected (before the sleep) The totalSleepTime includes the sleep duration for the upcoming sleep Note: called while holding the lock.

type OnSingleLimitExceeded

type OnSingleLimitExceeded func(*CallbackContext)

OnSingleLimitPassed is a callback to be called when a rate limit is exceeding the limit for a single sleep. The sleepUntil represents the end of sleep duration if the limit was not exceeded. The totalSleepTime does not include the sleep (that is not going to happen). Note: called while holding the lock.

type OnTotalLimitExceeded

type OnTotalLimitExceeded func(*CallbackContext)

OnTotalLimitExceeded is a callback to be called when a rate limit is exceeding the limit for the total sleep. The sleepUntil represents the end of sleep duration if the limit was not exceeded. The totalSleepTime does not include the sleep (that is not going to happen). Note: called while holding the lock.

type Option

type Option func(*SecondaryRateLimitConfig)

func GetConfigOverrides

func GetConfigOverrides(ctx context.Context) []Option

GetConfigOverrides returns the config overrides from the context, if any.

func WithLimitDetectedCallback

func WithLimitDetectedCallback(callback OnLimitDetected) Option

WithLimitDetectedCallback adds a callback to be called when a new active rate limit is detected.

func WithSingleSleepLimit

func WithSingleSleepLimit(limit time.Duration, callback OnSingleLimitExceeded) Option

WithSingleSleepLimit adds a limit to the duration allowed to wait for a single sleep (rate limit). The callback parameter is nillable.

func WithTotalSleepLimit

func WithTotalSleepLimit(limit time.Duration, callback OnTotalLimitExceeded) Option

WithTotalSleepLimit adds a limit to the accumulated duration allowed to wait for all sleeps (one or more rate limits). The callback parameter is nillable.

type SecondaryRateLimitBody

type SecondaryRateLimitBody struct {
	Message     string `json:"message"`
	DocumentURL string `json:"documentation_url"`
}

func (SecondaryRateLimitBody) IsSecondaryRateLimit

func (s SecondaryRateLimitBody) IsSecondaryRateLimit() bool

IsSecondaryRateLimit checks whether the response is a legitimate secondary rate limit. It checks the prefix of the message and the suffix of the documentation URL in the response body in case the message or documentation URL is modified in the future. https://docs.github.com/en/rest/overview/rate-limits-for-the-rest-api#about-secondary-rate-limits

type SecondaryRateLimitConfig

type SecondaryRateLimitConfig struct {
	// contains filtered or unexported fields
}

SecondaryRateLimitConfig is the config for the secondary rate limit waiter. Use the options to set the config.

func (*SecondaryRateLimitConfig) ApplyOptions

func (c *SecondaryRateLimitConfig) ApplyOptions(opts ...Option)

ApplyOptions applies the options to the config.

func (*SecondaryRateLimitConfig) IsAboveSingleSleepLimit

func (c *SecondaryRateLimitConfig) IsAboveSingleSleepLimit(sleepTime time.Duration) bool

IsAboveSingleSleepLimit returns true if the single sleep duration is above the limit.

func (*SecondaryRateLimitConfig) IsAboveTotalSleepLimit

func (c *SecondaryRateLimitConfig) IsAboveTotalSleepLimit(sleepTime time.Duration, totalSleepTime time.Duration) bool

IsAboveTotalSleepLimit returns true if the total sleep duration is above the limit.

type SecondaryRateLimitWaiter

type SecondaryRateLimitWaiter struct {
	Base http.RoundTripper
	// contains filtered or unexported fields
}

func NewRateLimitWaiter

func NewRateLimitWaiter(base http.RoundTripper, opts ...Option) (*SecondaryRateLimitWaiter, error)

func (*SecondaryRateLimitWaiter) RoundTrip

func (t *SecondaryRateLimitWaiter) RoundTrip(request *http.Request) (*http.Response, error)

RoundTrip handles the secondary rate limit by waiting for it to finish before issuing new requests. If a request got a secondary rate limit error as a response, we retry the request after waiting. Issuing more requests during a secondary rate limit may cause a ban from the server side, so we want to prevent these requests, not just for the sake of cpu/network utilization. Nonetheless, there is no way to prevent subtle race conditions without completely serializing the requests, so we prefer to let some slip in case of a race condition, i.e., after a retry-after response is received and before it is processed, a few other (concurrent) requests may be issued.