Documentation ¶
Overview ¶
Package galaxycache provides a data loading mechanism with caching and de-duplication that works across a set of peer processes.
Each data Get first consults its local cache, otherwise delegates to the requested key's canonical owner, which then checks its cache or finally gets the data. In the common case, many concurrent cache misses across a set of peers for the same key result in just one cache fill.
Index ¶
- Variables
- type AtomicInt
- type BackendGetter
- type ByteCodec
- type CacheStats
- type CacheType
- type Codec
- type CopyingByteCodec
- type FetchProtocol
- type Galaxy
- type GalaxyOption
- type GalaxyStats
- type GetterFunc
- type HCStatsWithTime
- type HashOptions
- type NullFetchProtocol
- type Peer
- type PeerPicker
- type RemoteFetcher
- type StringCodec
- type Universe
- func (universe *Universe) AddPeer(peer Peer) error
- func (universe *Universe) GetGalaxy(name string) *Galaxy
- func (universe *Universe) ListPeers() map[string]RemoteFetcher
- func (universe *Universe) NewGalaxy(name string, cacheBytes int64, getter BackendGetter, opts ...GalaxyOption) *Galaxy
- func (universe *Universe) RemovePeers(ids ...string) error
- func (universe *Universe) Set(peerURLs ...string) error
- func (universe *Universe) SetIncludeSelf(incSelf bool)
- func (universe *Universe) SetPeers(peers ...Peer) error
- func (universe *Universe) Shutdown() error
- type UniverseOpt
Constants ¶
This section is empty.
Variables ¶
var ( MGets = stats.Int64("galaxycache/gets", "The number of Get requests", stats.UnitDimensionless) MLoads = stats.Int64("galaxycache/loads", "The number of gets/cacheHits", stats.UnitDimensionless) MLoadErrors = stats.Int64("galaxycache/loads_errors", "The number of errors encountered during Get", stats.UnitDimensionless) MCacheHits = stats.Int64("galaxycache/cache_hits", "The number of times that the cache was hit", stats.UnitDimensionless) MPeerLoads = stats.Int64("galaxycache/peer_loads", "The number of remote loads or remote cache hits", stats.UnitDimensionless) MPeerLoadErrors = stats.Int64("galaxycache/peer_errors", "The number of remote errors", stats.UnitDimensionless) MBackendLoads = stats.Int64("galaxycache/backend_loads", "The number of successful loads from the backend getter", stats.UnitDimensionless) MBackendLoadErrors = stats.Int64("galaxycache/local_load_errors", "The number of failed backend loads", stats.UnitDimensionless) MCoalescedLoads = stats.Int64("galaxycache/coalesced_loads", "The number of loads coalesced by singleflight", stats.UnitDimensionless) MCoalescedCacheHits = stats.Int64("galaxycache/coalesced_cache_hits", "The number of coalesced times that the cache was hit", stats.UnitDimensionless) MCoalescedPeerLoads = stats.Int64("galaxycache/coalesced_peer_loads", "The number of coalesced remote loads or remote cache hits", stats.UnitDimensionless) MCoalescedBackendLoads = stats.Int64("galaxycache/coalesced_backend_loads", "The number of coalesced successful loads from the backend getter", stats.UnitDimensionless) MServerRequests = stats.Int64("galaxycache/server_requests", "The number of Gets that came over the network from peers", stats.UnitDimensionless) MKeyLength = stats.Int64("galaxycache/key_length", "The length of keys", stats.UnitBytes) MValueLength = stats.Int64("galaxycache/value_length", "The length of values", stats.UnitBytes) MRoundtripLatencyMilliseconds = stats.Float64("galaxycache/roundtrip_latency", "Roundtrip latency in milliseconds", stats.UnitMilliseconds) MCacheSize = stats.Int64("galaxycache/cache_bytes", "The number of bytes used for storing Keys and Values in the cache", stats.UnitBytes) MCacheEntries = stats.Int64("galaxycache/cache_entries", "The number of entries in the cache", stats.UnitDimensionless) MGetterFuncLatencyMilliseconds = stats.Float64("galaxycache/getterfunc_latency", "Local getter function latency in milliseconds", stats.UnitMilliseconds) )
Opencensus stats
var ( // GalaxyKey tags the name of the galaxy GalaxyKey = tag.MustNewKey("galaxy") // CacheLevelKey tags the level at which data was found on Get CacheLevelKey = tag.MustNewKey("cache-hit-level") // CacheTypeKey tags the galaxy sub-cache the metric applies to CacheTypeKey = tag.MustNewKey("cache-type") )
var AllViews = []*view.View{ {Measure: MGets, TagKeys: []tag.Key{GalaxyKey}, Aggregation: view.Count()}, {Measure: MLoads, TagKeys: []tag.Key{GalaxyKey}, Aggregation: view.Count()}, {Measure: MCacheHits, TagKeys: []tag.Key{GalaxyKey, CacheLevelKey}, Aggregation: view.Count()}, {Measure: MPeerLoads, TagKeys: []tag.Key{GalaxyKey}, Aggregation: view.Count()}, {Measure: MPeerLoadErrors, TagKeys: []tag.Key{GalaxyKey}, Aggregation: view.Count()}, {Measure: MBackendLoads, TagKeys: []tag.Key{GalaxyKey}, Aggregation: view.Count()}, {Measure: MBackendLoadErrors, TagKeys: []tag.Key{GalaxyKey}, Aggregation: view.Count()}, {Measure: MCoalescedLoads, TagKeys: []tag.Key{GalaxyKey}, Aggregation: view.Count()}, {Measure: MCoalescedCacheHits, TagKeys: []tag.Key{GalaxyKey, CacheLevelKey}, Aggregation: view.Count()}, {Measure: MCoalescedPeerLoads, TagKeys: []tag.Key{GalaxyKey}, Aggregation: view.Count()}, {Measure: MCoalescedBackendLoads, TagKeys: []tag.Key{GalaxyKey}, Aggregation: view.Count()}, {Measure: MServerRequests, TagKeys: []tag.Key{GalaxyKey}, Aggregation: view.Count()}, {Measure: MKeyLength, TagKeys: []tag.Key{GalaxyKey}, Aggregation: defaultBytesDistribution}, {Measure: MValueLength, TagKeys: []tag.Key{GalaxyKey}, Aggregation: defaultBytesDistribution}, {Measure: MRoundtripLatencyMilliseconds, TagKeys: []tag.Key{GalaxyKey}, Aggregation: defaultMillisecondsDistribution}, {Measure: MCacheSize, TagKeys: []tag.Key{GalaxyKey, CacheTypeKey}, Aggregation: view.LastValue()}, {Measure: MCacheEntries, TagKeys: []tag.Key{GalaxyKey, CacheTypeKey}, Aggregation: view.LastValue()}, {Measure: MGetterFuncLatencyMilliseconds, TagKeys: []tag.Key{GalaxyKey}, Aggregation: defaultMillisecondsDistribution}, }
AllViews is a slice of default views for people to use
Functions ¶
This section is empty.
Types ¶
type AtomicInt ¶
type AtomicInt struct {
// contains filtered or unexported fields
}
An AtomicInt is an int64 to be accessed atomically. It thinly wraps atomic.Int64 with go1.19+
type BackendGetter ¶
type BackendGetter interface { // Get populates dest with the value identified by key // // The returned data must be unversioned. That is, key must // uniquely describe the loaded data, without an implicit // current time, and without relying on cache expiration // mechanisms. Get(ctx context.Context, key string, dest Codec) error }
A BackendGetter loads data for a key.
type ByteCodec ¶
type ByteCodec []byte
ByteCodec is a byte slice type that implements Codec
func (*ByteCodec) MarshalBinary ¶
MarshalBinary on a ByteCodec returns the bytes
func (*ByteCodec) UnmarshalBinary ¶
UnmarshalBinary on a ByteCodec sets the ByteCodec to a copy of the provided data
type CacheStats ¶
CacheStats are returned by stats accessors on Galaxy.
type CacheType ¶
type CacheType uint8
CacheType represents a type of cache.
const ( // MainCache is the cache for items that this peer is the // owner of. MainCache CacheType = iota + 1 // HotCache is the cache for items that seem popular // enough to replicate to this node, even though it's not the // owner. HotCache // CandidateCache is the cache for peer-owned keys that // may become popular enough to put in the HotCache CandidateCache )
type CopyingByteCodec ¶
type CopyingByteCodec []byte
CopyingByteCodec is a byte slice type that implements Codec and returns a copy of the bytes when marshaled
func (*CopyingByteCodec) MarshalBinary ¶
func (c *CopyingByteCodec) MarshalBinary() ([]byte, error)
MarshalBinary on a CopyingByteCodec returns a copy of the bytes
func (*CopyingByteCodec) UnmarshalBinary ¶
func (c *CopyingByteCodec) UnmarshalBinary(data []byte) error
UnmarshalBinary on a CopyingByteCodec sets the ByteCodec to a copy of the provided data
type FetchProtocol ¶
type FetchProtocol interface { // NewFetcher instantiates the connection between the current and a // remote peer and returns a RemoteFetcher to be used for fetching // data from that peer NewFetcher(url string) (RemoteFetcher, error) }
FetchProtocol defines the chosen fetching protocol to peers (namely HTTP or GRPC) and implements the instantiation method for that connection (creating a new RemoteFetcher)
type Galaxy ¶
type Galaxy struct { // Stats are statistics on the galaxy. Stats GalaxyStats // contains filtered or unexported fields }
A Galaxy is a cache namespace and associated data spread over a group of 1 or more machines.
func (*Galaxy) CacheStats ¶
func (g *Galaxy) CacheStats(which CacheType) CacheStats
CacheStats returns stats about the provided cache within the galaxy.
func (*Galaxy) Get ¶
Get as defined here is the primary "get" called on a galaxy to find the value for the given key, using the following logic: - First, try the local cache; if its a cache hit, we're done - On a cache miss, search for which peer is the owner of the key based on the consistent hash - If a different peer is the owner, use the corresponding fetcher to Fetch from it; otherwise, if the calling instance is the key's canonical owner, call the BackendGetter to retrieve the value (which will now be cached locally)
type GalaxyOption ¶
type GalaxyOption interface {
// contains filtered or unexported methods
}
GalaxyOption is an interface for implementing functional galaxy options
func WithClock ¶
func WithClock(clk clocks.Clock) GalaxyOption
WithClock lets one override the clock used internally for key-stats accounting (among other things).
func WithHotCacheRatio ¶
func WithHotCacheRatio(r int64) GalaxyOption
WithHotCacheRatio allows the client to specify a ratio for the main-to-hot cache sizes for the galaxy; defaults to 8:1
func WithIdleStatsAgeResetWindow ¶
func WithIdleStatsAgeResetWindow(age time.Duration) GalaxyOption
WithIdleStatsAgeResetWindow overrides the default interval after which a key that's been idle for a while gets its stats reset (such that that hit is recorded as if it were the first).
func WithMaxCandidates ¶
func WithMaxCandidates(n int) GalaxyOption
WithMaxCandidates allows the client to specify the size of the candidate cache by the max number of candidates held at one time; defaults to 100
func WithPromoter ¶
func WithPromoter(p promoter.Interface) GalaxyOption
WithPromoter allows the client to specify a promoter for the galaxy; defaults to a simple QPS comparison
type GalaxyStats ¶
type GalaxyStats struct { Gets AtomicInt // any Get request, including from peers Loads AtomicInt // (gets - cacheHits) CoalescedLoads AtomicInt // inside singleflight MaincacheHits AtomicInt // number of maincache hits HotcacheHits AtomicInt // number of hotcache hits PeerLoads AtomicInt // either remote load or remote cache hit (not an error) PeerLoadErrors AtomicInt // errors on getFromPeer BackendLoads AtomicInt // load from backend locally BackendLoadErrors AtomicInt // total bad local loads CoalescedMaincacheHits AtomicInt // maincache hit in singleflight CoalescedHotcacheHits AtomicInt // hotcache hit in singleflight CoalescedPeerLoads AtomicInt // peer load in singleflight CoalescedBackendLoads AtomicInt // backend load in singleflight ServerRequests AtomicInt // gets that came over the network from peers }
GalaxyStats are per-galaxy statistics.
type GetterFunc ¶
A GetterFunc implements BackendGetter with a function.
type HCStatsWithTime ¶
type HCStatsWithTime struct {
// contains filtered or unexported fields
}
HCStatsWithTime includes a time stamp along with the hotcache stats to ensure updates happen no more than once per second
type HashOptions ¶
type HashOptions struct { // Replicas specifies the number of key replicas on the consistent hash. // If zero, it defaults to 50. Replicas int // HashFn specifies the hash function of the consistent hash. // If nil, it defaults to crc32.ChecksumIEEE. HashFn consistenthash.Hash }
HashOptions specifies the the hash function and the number of replicas for consistent hashing
type NullFetchProtocol ¶
type NullFetchProtocol struct{}
NullFetchProtocol implements FetchProtocol, but always returns errors. (useful for unit-testing)
func (*NullFetchProtocol) NewFetcher ¶
func (n *NullFetchProtocol) NewFetcher(url string) (RemoteFetcher, error)
NewFetcher instantiates the connection between the current and a remote peer and returns a RemoteFetcher to be used for fetching data from that peer
type Peer ¶
type Peer struct { // Unique ID for this peer (e.g. in k8s may be a pod name) ID string // URI or URL that the registered PeerFetcher can connect to // URI should be a valid base URL, // for example "example.net:8000" or "10.32.54.231:8123". URI string }
Peer is an ID and ip:port/url tuple for a specific peer
type PeerPicker ¶
type PeerPicker struct {
// contains filtered or unexported fields
}
PeerPicker is in charge of dealing with peers: it contains the hashing options (hash function and number of replicas), consistent hash map of peers, and a map of RemoteFetchers to those peers
type RemoteFetcher ¶
type RemoteFetcher interface { Fetch(context context.Context, galaxy string, key string) ([]byte, error) // Close closes a client-side connection (may be a nop) Close() error }
RemoteFetcher is the interface that must be implemented to fetch from other peers; the PeerPicker contains a map of these fetchers corresponding to each other peer address
type StringCodec ¶
type StringCodec string
StringCodec is a string type that implements Codec
func (*StringCodec) MarshalBinary ¶
func (c *StringCodec) MarshalBinary() ([]byte, error)
MarshalBinary on a StringCodec returns the bytes underlying the string
func (*StringCodec) UnmarshalBinary ¶
func (c *StringCodec) UnmarshalBinary(data []byte) error
UnmarshalBinary on a StringCodec sets the StringCodec to a stringified copy of the provided data
type Universe ¶
type Universe struct {
// contains filtered or unexported fields
}
Universe defines the primary container for all galaxycache operations. It contains the galaxies and PeerPicker
func NewUniverse ¶
func NewUniverse(protocol FetchProtocol, selfID string, opts ...UniverseOpt) *Universe
NewUniverse is the main constructor for the Universe object. It is passed a FetchProtocol (to specify fetching via GRPC or HTTP) and its own URL along with options.
func NewUniverseWithOpts ¶
func NewUniverseWithOpts(protocol FetchProtocol, selfID string, options *HashOptions) *Universe
NewUniverseWithOpts is a deprecated constructor for the Universe object that defines a non-default hash function and number of replicas. Please use `NewUniverse` with the `WithHashOpts` option instead.
func (*Universe) AddPeer ¶
AddPeer updates the Universe's list of peers to include the passed peer (contained in the PeerPicker). The Peer's URI value should be a valid base URL, while the ID may be anything that's unique, for example "example.net:8000". If Set and AddPeer are mixed, the ID and URI fields must match.
func (*Universe) GetGalaxy ¶
GetGalaxy returns the named galaxy previously created with NewGalaxy, or nil if there's no such galaxy.
func (*Universe) ListPeers ¶
func (universe *Universe) ListPeers() map[string]RemoteFetcher
ListPeers returns a map of remote fetchers keyed by Peer ID, useful for testing incremental changes to galaxycache peers.
func (*Universe) NewGalaxy ¶
func (universe *Universe) NewGalaxy(name string, cacheBytes int64, getter BackendGetter, opts ...GalaxyOption) *Galaxy
NewGalaxy creates a coordinated galaxy-aware BackendGetter from a BackendGetter.
The returned BackendGetter tries (but does not guarantee) to run only one Get is called once for a given key across an entire set of peer processes. Concurrent callers both in the local process and in other processes receive copies of the answer once the original Get completes.
The galaxy name must be unique for each BackendGetter.
func (*Universe) RemovePeers ¶
RemovePeers updates the Universe's list of peers to remove the passed peers IDs (contained in the PeerPicker). The arguments should match the ID field on SetPeers and AddPeers calls and the URLs passed to Set. unrecognized IDs are ignored
func (*Universe) Set ¶
Set updates the Universe's list of peers (contained in the PeerPicker). Each PeerURL value should be a valid base URL, for example "example.net:8000". This is a compatibility wrapper around SetPeers which sets the ID and URI equal.
func (*Universe) SetIncludeSelf ¶
SetIncludeSelf toggles the inclusion of the "self ID" for the universe in the PeerPicker's hash-ring
type UniverseOpt ¶
type UniverseOpt func(*universeOpts)
UniverseOpt is a functional Universe option.
func WithHashOpts ¶
func WithHashOpts(hashOpts *HashOptions) UniverseOpt
WithHashOpts sets the HashOptions on a universe.
func WithRecorder ¶
func WithRecorder(recorder stats.Recorder) UniverseOpt
WithRecorder allows you to override the default stats.Recorder used for stats.
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
Package consistenthash provides an implementation of a ring hash.
|
Package consistenthash provides an implementation of a ring hash. |
Package lru implements an LRU cache.
|
Package lru implements an LRU cache. |
Package singleflight provides a duplicate function call suppression mechanism.
|
Package singleflight provides a duplicate function call suppression mechanism. |