cache

Documentation

Overview

Package cache provides an LRU cache implementation to be used by the RLS LB policy to cache RLS response data.

Index

• type BackoffState
• type Entry
• type Key
• type LRU
  • func NewLRU(maxSize int64, onEvicted func(Key, *Entry)) *LRU
  • func (lru *LRU) Add(key Key, value *Entry)
  • func (lru *LRU) Get(key Key) *Entry
  • func (lru *LRU) Remove(key Key)
  • func (lru *LRU) Resize(newMaxSize int64)

Types

type BackoffState

type BackoffState struct {
	// Retries keeps track of the number of RLS failures, to be able to
	// determine the amount of time to backoff before the next attempt.
	Retries int
	// Backoff is an exponential backoff implementation which returns the
	// amount of time to backoff, given the number of retries.
	Backoff backoff.Strategy
	// Timer fires when the backoff period ends and incoming requests after
	// this will trigger a new RLS request.
	Timer *time.Timer
	// Callback provided by the LB policy to be notified when the backoff timer
	// expires. This will trigger a new picker to be returned to the
	// ClientConn, to force queued up RPCs to be retried.
	Callback func()
}

BackoffState wraps all backoff related state associated with a cache entry.

type Entry

type Entry struct {
	// Mu synchronizes access to this particular cache entry. The LB policy
	// will also hold another mutex to synchronize access to the cache as a
	// whole. To avoid holding the top-level mutex for the whole duration for
	// which one particular cache entry is acted upon, we use this entry mutex.
	Mu sync.Mutex
	// ExpiryTime is the absolute time at which the data cached as part of this
	// entry stops being valid. When an RLS request succeeds, this is set to
	// the current time plus the max_age field from the LB policy config. An
	// entry with this field in the past is not used to process picks.
	ExpiryTime time.Time
	// BackoffExpiryTime is the absolute time at which an entry which has gone
	// through backoff stops being valid.  When an RLS request fails, this is
	// set to the current time plus twice the backoff time. The cache expiry
	// timer will only delete entries for which both ExpiryTime and
	// BackoffExpiryTime are in the past.
	BackoffExpiryTime time.Time
	// StaleTime is the absolute time after which this entry will be
	// proactively refreshed if we receive a request for it. When an RLS
	// request succeeds, this is set to the current time plus the stale_age
	// from the LB policy config.
	StaleTime time.Time
	// BackoffTime is the absolute time at which the backoff period for this
	// entry ends. The backoff timer is setup with this value. No new RLS
	// requests are sent out for this entry until the backoff period ends.
	BackoffTime time.Time
	// EarliestEvictTime is the absolute time before which this entry should
	// not be evicted from the cache. This is set to a default value of 5
	// seconds when the entry is created. This is required to make sure that a
	// new entry added to the cache is not evicted before the RLS response
	// arrives (usually when the cache is too small).
	EarliestEvictTime time.Time
	// CallStatus stores the RPC status of the previous RLS request for this
	// entry. Picks for entries with a non-nil value for this field are failed
	// with the error stored here.
	CallStatus error
	// Backoff contains all backoff related state. When an RLS request
	// succeeds, backoff state is reset.
	Backoff BackoffState
	// HeaderData is received in an RLS response and is to be sent in the
	// X-Google-RLS-Data header for matching RPCs.
	HeaderData string
	// ChildPicker is a very thin wrapper around the child policy wrapper.
	// The type is declared as a Picker interface since the users of
	// the cache only care about the picker provided by the child policy, and
	// this makes it easy for testing.
	ChildPicker balancer.Picker
	// contains filtered or unexported fields
}

Entry wraps all the data to be stored in a cache entry.

type Key

type Key struct {
	// Path is the full path of the incoming RPC request.
	Path string
	// KeyMap is a stringified version of the RLS request keys built using the
	// RLS keyBuilder. Since map is not a Type which is comparable in Go, it
	// cannot be part of the key for another map (the LRU cache is implemented
	// using a native map type).
	KeyMap string
}

Key represents the cache key used to uniquely identify a cache entry.

type LRU

type LRU struct {
	// contains filtered or unexported fields
}

LRU is a cache with a least recently used eviction policy. It is not safe for concurrent access.

func NewLRU

func NewLRU(maxSize int64, onEvicted func(Key, *Entry)) *LRU

NewLRU creates a cache.LRU with a size limit of maxSize and the provided eviction callback.

Currently, only the cache.Key and the HeaderData field from cache.Entry count towards the size of the cache (other overhead per cache entry is not counted). The cache could temporarily exceed the configured maxSize because we want the entries to spend a configured minimum amount of time in the cache before they are LRU evicted (so that all the work performed in sending an RLS request and caching the response is not a total waste).

The provided onEvited callback must not attempt to re-add the entry inline and the RLS LB policy does not have a need to do that.

The cache package trusts the RLS policy (its only user) to supply a default minimum non-zero maxSize, in the event that the ServiceConfig does not provide a value for it.

func (*LRU) Add

func (lru *LRU) Add(key Key, value *Entry)

Add adds a new entry to the cache.

func (*LRU) Get

func (lru *LRU) Get(key Key) *Entry

Get returns a cache entry with key key.

func (*LRU) Remove

func (lru *LRU) Remove(key Key)

Remove removes a cache entry wth key key, if one exists.

func (*LRU) Resize

func (lru *LRU) Resize(newMaxSize int64)

Resize sets the size limit of the LRU to newMaxSize and removes older entries, if required, to comply with the new limit.