mnemosyne

package module
v0.0.0-...-b2a7555 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 15, 2020 License: MIT Imports: 18 Imported by: 0

README

Mnemosyne

Mnemosyne

Mnemosyne (/nɪˈmɒzɪniː, nɪˈmɒsɪniː/; Greek: Μνημοσύνη, pronounced [mnɛːmosýːnɛː]) is a multilayer cache which can be configured to use various configurations of Redis and/or BigCache (an in-memory caching package) with minimum configuration out of the box.

Getting Started

Installing
go get -u github.com/cafebazaar/mnemosyne
Initializing a Manager and Selecting a Cache Instance
mnemosyneManager := mnemosyne.NewMnemosyne(config, nil, nil)
cacheInstance := mnemosyneManager.select("result-cache")
Working with a CacheInstance
  cacheInstance.Set(context, key, value)
  var myCachedData myType
  err := cacheInstance.Get(context, key, &myCachedData)
  // remember: cacheMiss is also an Error

Configuration

Mnemosyne uses Viper as it's config engine. Template of each cache instance includes the list of the layers' names (in order of precedence) followed by configuration for each layer. here's an example:

cache:
  my-result-cache: # arbitary name for the cache instance
    soft-ttl: 2h 
    layers:   # Arbitary names for each cache layer
      - result-memory
      - result-gaurdian
    result-memory:
      type: memory
      max-memory: 512
      ttl: 2h
      amnesia: 10
      compression: true
    result-gaurdian:
      type: gaurdian
      address: "localhost:6379"
      slaves:
        - "localhost:6380"
        - "localhost:6381"
      db: 2
      ttl: 24h
      amnesia: 0
      compression: true

  my-user-cache:
    soft-ttl: 2h
    layers:
      - user-memory
      - user-redis
    user-memory:
      type: memory
      max-memory: 512
      ttl: 2h
      amnesia: 0
      compression: true
    user-redis:
      type: redis
      address: "localhost:6379"
      db: 4
      ttl: 24h
      amnesia: 0
      compression: true

Each cache layer can be of the following types:

redis is used for a single node Redis server.

gaurdian [Depricated] is used for a master-slave Redis cluster configuration but it's being depricated in favor of rediscluster.

rediscluster is an all-encompassing configuration for both client side sharding as well as cluster Redis (or both at the same time).

memory uses the BigCache library to provide an efficient and fast in-memory cache.

tiny uses the native sync.map data structure to store smaller cache values in memory (used for low-write caches).

Note: all of the cache types are sync-safe, meaning they can be safely used from simultaneously running goroutines.

Instance Configs:

soft-ttl is an instance-wide TTL which when expired will NOT remove the data from the instance, but warns that the data is old.

Common Layer Configs:

amnesia is a stochastic fall-through mechanism which allows for a higher layer to be updated from a lower layer by the way of an artificial cache-miss, an amnesia value of 0 means that the layers will never miss a data that they actually have, an amnesia value of 10 means when a key is present in the cache, 90% of the time it is returned but 10% of the time it is ignored and is treated as a cache-miss. a 100% amnesia effectively turns the layer off. (Default: 0)
Note: 'SET' operations ignore Amnesia, to compeletly turn off a layer, remove its name from the layer list.

compression dictates whther the data is compressed before being put into the cache memory. Currently only Zlib compression is supported. (Default: false)

ttl is the hard Time-To-Live for the data in this particular layer, after which the data is expired and is expected to be removed.

Type-spesific Layer Configs:

db {redis - gaurdian} is the Redis DB number to be used. (Default:0)
idle-timeout {redis - gaurdian} is the timeout for idle connections to the Redis Server (see Redis documentation) (Default:0 - no timeout)
address {redis - gaurdian - rediscluster} is the Redis Server's Address (the master's address in case of a cluster)
slaves {gaurdian - rediscluster} is a list of Redis servers addresses pertaining to the slave nodes.
max-memory {memory} is the maximum amount of system memory which can be used by this particular layer.

Epimetheus Integration Guide

Add these two functions to your container.go file as well as to the wire.build() so wire-gen can recognize the proper timer & counter to pass to Mnemosyne.

func getCommTimer(epi *epimetheus.Epimetheus) mnemosyne.ITimer {
	return epi.CommTimer
}

func getCacheRate(epi *epimetheus.Epimetheus) mnemosyne.ICounter {
	return epi.CacheRate
}

Documentation

Documents are available at https://godoc.org/github.com/cafebazaar/mnemosyne

Built With

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository.

Roadmap

- Improve documentation
- Add tests

Authors

  • Ramtin Rostami rrostami - Initial work & Maintaining
  • Pedram Teymoori pedramteymoori - Initial work & Maintaining
  • Parsa abdollahi - Initial work
  • Ava Abderezaei avv-va - Tests

See also the list of contributors who participated in this project.

License

This project is licensed under the MIT License - see the LICENSE.md file for details

Acknowledgments

Made with ❤ in Cafebazaar Search Team

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func MakeKey 

func MakeKey(keys ...string) string

func NewFastMemoryCache 

func NewFastMemoryCache(opts *CacheOpts) *fastMemoryCache

func NewInMemoryCache 

func NewInMemoryCache(opts *CacheOpts) *inMemoryCache

func NewShardedClusterRedisCache 

func NewShardedClusterRedisCache(opts *CacheOpts, watcher ITimer) *redisCache

func NewTinyCache 

func NewTinyCache(opts *CacheOpts) *tinyCache

Types

type CacheOpts 

type CacheOpts struct {
	// contains filtered or unexported fields
}

type DummyCounter 

type DummyCounter struct {
}

func NewDummyCounter 

func NewDummyCounter() *DummyCounter

func (*DummyCounter) Inc 

func (*DummyCounter) Inc(...string)

type DummyTimer 

type DummyTimer struct {
}

func NewDummyTimer 

func NewDummyTimer() *DummyTimer

func (*DummyTimer) Done 

func (*DummyTimer) Done(a time.Time, b ...string)

func (*DummyTimer) Start 

func (*DummyTimer) Start() time.Time

type ErrCacheMiss 

type ErrCacheMiss struct {
	// contains filtered or unexported fields
}

ErrCacheMiss is the Error returned when a cache miss happens

func (*ErrCacheMiss) Error 

func (e *ErrCacheMiss) Error() string

type ICache 

type ICache interface {
	Get(context.Context, string, interface{}) (*cachable, error)
	Set(context.Context, string, *cachable) error
	Delete(context.Context, string) error
	Clear() error
	TTL(context.Context, string) time.Duration
	Name() string
}

func NewCacheLayer 

func NewCacheLayer(opts *CacheOpts, watcher ITimer) ICache

type ICounter 

type ICounter interface {
	Inc(...string)
}

type ITimer 

type ITimer interface {
	Start() time.Time
	Done(time.Time, ...string)
}

type MemoryOpts 

type MemoryOpts struct {
	// contains filtered or unexported fields
}

type Mnemosyne 

type Mnemosyne struct {
	// contains filtered or unexported fields
}

Mnemosyne is the parent object which holds all cache instances

func NewMnemosyne 

func NewMnemosyne(config *viper.Viper, commTimer ITimer, cacheHitCounter ICounter) *Mnemosyne

NewMnemosyne initializes the Mnemosyne object which holds all the cache instances

func (*Mnemosyne) Select 

func (m *Mnemosyne) Select(cacheName string) *MnemosyneInstance

Select returns a cache instance selected by name

type MnemosyneInstance 

type MnemosyneInstance struct {
	// contains filtered or unexported fields
}

MnemosyneInstance is an instance of a multi-layer cache

func (*MnemosyneInstance) Delete 

func (mn *MnemosyneInstance) Delete(ctx context.Context, key string) error

Delete removes a key from all the layers (if exists)

func (*MnemosyneInstance) Flush 

func (mn *MnemosyneInstance) Flush(targetLayerName string) error

Flush completly clears a single layer of the cache

func (*MnemosyneInstance) Get 

func (mn *MnemosyneInstance) Get(ctx context.Context, key string, refrence interface{}) (interface{}, error)

Get retrieves the value for key

func (*MnemosyneInstance) GetAndShouldUpdate 

func (mn *MnemosyneInstance) GetAndShouldUpdate(ctx context.Context, key string, refrence interface{}) (interface{}, bool, error)

GetAndShouldUpdate retrieves the value for key and also shows whether the soft-TTL of that key has passed or not

func (*MnemosyneInstance) Set 

func (mn *MnemosyneInstance) Set(ctx context.Context, key string, value interface{}) error

Set sets the value for a key in all layers of the cache instance

func (*MnemosyneInstance) ShouldUpdate 

func (mn *MnemosyneInstance) ShouldUpdate(ctx context.Context, key string) (bool, error)

ShouldUpdate shows whether the soft-TTL of a key has passed or not

func (*MnemosyneInstance) ShouldUpdateDeep 

func (mn *MnemosyneInstance) ShouldUpdateDeep(ctx context.Context, key string, refrence interface{}) (bool, error)

ShouldUpdateDeep checks all layers for newer result and will sync older cache layers

func (*MnemosyneInstance) TTL 

func (mn *MnemosyneInstance) TTL(ctx context.Context, key string) (int, time.Duration)

TTL returns the TTL of the first accessible data instance as well as the layer it was found on

type RedisClusterAddress 

type RedisClusterAddress struct {
	MasterAddr string   `mapstructure:"address"`
	SlaveAddrs []string `mapstructure:"slaves"`
}

type RedisOpts 

type RedisOpts struct {
	// contains filtered or unexported fields
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.