Documentation
¶
Index ¶
Constants ¶
View Source
const ( DefaultDialTimeout = 5 * time.Second DefaultKeepalive = 5 * time.Minute DefaultReadTimeout = 3 * time.Second DefaultIdleTimeout = 5 * time.Minute DefaultMinRetryBackoff = 8 * time.Millisecond DefaultMaxRetryBackoff = 512 * time.Millisecond DefaultMaxRetries = 3 )
View Source
const ( // SyncReplicationMode enables sync replication mode which means that the // caller is blocked until write/delete operation is applied by replica // owners. The default mode is SyncReplicationMode SyncReplicationMode = 0 // AsyncReplicationMode enables async replication mode which means that // write/delete operations are done in a background task. AsyncReplicationMode = 1 )
View Source
const ( LogLevelDebug = "DEBUG" LogLevelWarn = "WARN" LogLevelError = "ERROR" LogLevelInfo = "INFO" )
View Source
const ( // DefaultPort is for Olric DefaultPort = 3320 // DefaultDiscoveryPort is for memberlist DefaultDiscoveryPort = 3322 // DefaultPartitionCount denotes default partition count in the cluster. DefaultPartitionCount = 271 // DefaultLoadFactor is used by the consistent hashing function. Keep it small. DefaultLoadFactor = 1.25 // DefaultLogLevel determines the log level without extra configuration. // It's DEBUG. DefaultLogLevel = LogLevelDebug // DefaultLogVerbosity denotes default log verbosity level. // // * flog.V(1) - Generally useful for this to ALWAYS be visible to an operator // * Programmer errors // * Logging extra info about a panic // * CLI argument handling // * flog.V(2) - A reasonable default log level if you don't want verbosity. // * Information about config (listening on X, watching Y) // * Errors that repeat frequently that relate to conditions that can be // corrected (pod detected as unhealthy) // * flog.V(3) - Useful steady state information about the service and // important log messages that may correlate to // significant changes in the system. This is the recommended default log // level for most systems. // * Logging HTTP requests and their exit code // * System state changing (killing pod) // * Controller state change events (starting pods) // * Scheduler log messages // * flog.V(4) - Extended information about changes // * More info about system state changes // * flog.V(5) - Debug level verbosity // * Logging in particularly thorny parts of code where you may want to come // back later and check it // * flog.V(6) - Trace level verbosity // * Context to understand the steps leading up to neterrors and warnings // * More information for troubleshooting reported issues DefaultLogVerbosity = 3 // MinimumReplicaCount denotes default and minimum replica count in an Olric // cluster. MinimumReplicaCount = 1 // DefaultBootstrapTimeout denotes default timeout value to check bootstrapping // status. DefaultBootstrapTimeout = 10 * time.Second // DefaultJoinRetryInterval denotes a time gap between sequential join attempts. DefaultJoinRetryInterval = time.Second // DefaultMaxJoinAttempts denotes a maximum number of failed join attempts // before forming a standalone cluster. DefaultMaxJoinAttempts = 10 // MinimumMemberCountQuorum denotes minimum required count of members to form // a cluster. MinimumMemberCountQuorum = 1 // DefaultLRUSamples is a sane default for randomly selected keys // in approximate LRU implementation. It's 5. DefaultLRUSamples int = 5 // LRUEviction assigns this as EvictionPolicy in order to enable LRU eviction // algorithm. LRUEviction EvictionPolicy = "LRU" // DefaultStorageEngine denotes the storage engine implementation provided by // Olric project. DefaultStorageEngine = "kvstore" // DefaultRoutingTablePushInterval is interval between routing table push events. DefaultRoutingTablePushInterval = time.Minute // DefaultTriggerBalancerInterval is interval between two sequential call of balancer worker. DefaultTriggerBalancerInterval = 15 * time.Second // DefaultCheckEmptyFragmentsInterval is the default value of interval between // two sequential call of empty fragment cleaner. It's one minute by default. DefaultCheckEmptyFragmentsInterval = time.Minute // DefaultTriggerCompactionInterval is the default value of interval between // two sequential call of compaction workers. The compaction worker works until // its work is done. It's 10 minutes by default. DefaultTriggerCompactionInterval = 10 * time.Minute // DefaultLeaveTimeout is the default value of maximum amount of time before DefaultLeaveTimeout = 5 * time.Second DefaultReadQuorum = 1 DefaultWriteQuorum = 1 DefaultMemberCountQuorum = 1 // DefaultKeepAlivePeriod is the default value of TCP keepalive. It's 300 seconds. // This option is useful in order to detect dead peers (clients that cannot // be reached even if they look connected). Moreover, if there is network // equipment between clients and servers that need to see some traffic in // order to take the connection open, the option will prevent unexpected // connection closed events. DefaultKeepAlivePeriod = 300 * time.Second )
Variables ¶
This section is empty.
Functions ¶
func NewMemberlistConfig ¶
func NewMemberlistConfig(env string) (*memberlist.Config, error)
NewMemberlistConfig returns a new memberlist.Config for a given environment.
It takes an env parameter: local, lan and wan.
local: DefaultLocalConfig works like DefaultConfig, however it returns a configuration that is optimized for a local loopback environments. The default configuration is still very conservative and errs on the side of caution.
lan: DefaultLANConfig returns a sane set of configurations for Memberlist. It uses the hostname as the node name, and otherwise sets very conservative values that are sane for most LAN environments. The default configuration errs on the side of caution, choosing values that are optimized for higher convergence at the cost of higher bandwidth usage. Regardless, these values are a good starting point when getting started with memberlist.
wan: DefaultWANConfig works like DefaultConfig, however it returns a configuration that is optimized for most WAN environments. The default configuration is still very conservative and errs on the side of caution.
Types ¶
type Client ¶ added in v0.3.0
type Client struct {
// Dial timeout for establishing new connections.
// Default is 5 seconds.
DialTimeout time.Duration
// Timeout for socket reads. If reached, commands will fail
// with a timeout instead of blocking. Use value -1 for no timeout and 0 for default.
// Default is 3 seconds.
ReadTimeout time.Duration
// Timeout for socket writes. If reached, commands will fail
// with a timeout instead of blocking.
// Default is ReadTimeout.
WriteTimeout time.Duration
// Dialer creates new network connection and has priority over
// Network and Addr options.
Dialer func(ctx context.Context, network, addr string) (net.Conn, error)
// Hook that is called when new connection is established.
OnConnect func(ctx context.Context, cn *redis.Conn) error
// Maximum number of retries before giving up.
// Default is 3 retries; -1 (not 0) disables retries.
MaxRetries int
// Minimum backoff between each retry.
// Default is 8 milliseconds; -1 disables backoff.
MinRetryBackoff time.Duration
// Maximum backoff between each retry.
// Default is 512 milliseconds; -1 disables backoff.
MaxRetryBackoff time.Duration
// Type of connection pool.
// true for FIFO pool, false for LIFO pool.
// Note that fifo has higher overhead compared to lifo.
PoolFIFO bool
// Maximum number of socket connections.
// Default is 10 connections per every available CPU as reported by runtime.GOMAXPROCS.
PoolSize int
// Minimum number of idle connections which is useful when establishing
// new connection is slow.
MinIdleConns int
// Connection age at which client retires (closes) the connection.
// Default is to not close aged connections.
MaxConnAge time.Duration
// Amount of time client waits for connection if all connections
// are busy before returning an error.
// Default is ReadTimeout + 1 second.
PoolTimeout time.Duration
// Amount of time after which client closes idle connections.
// Should be less than server's timeout.
// Default is 5 minutes. -1 disables idle timeout check.
IdleTimeout time.Duration
// Frequency of idle checks made by idle connections reaper.
// Default is 1 minute. -1 disables idle connections reaper,
// but idle connections are still discarded by the client
// if IdleTimeout is set.
IdleCheckFrequency time.Duration
// TLS Config to use. When set TLS will be negotiated.
TLSConfig *tls.Config
// Limiter interface used to implemented circuit breaker or rate limiter.
Limiter redis.Limiter
}
Client denotes configuration for TCP clients in Olric and the official Golang client.
func NewClient ¶ added in v0.3.0
func NewClient() *Client
NewClient returns a new configuration object for clients.
func (*Client) RedisOptions ¶ added in v0.5.0
func (c *Client) RedisOptions() *redis.Options
type Config ¶
type Config struct {
// Interface denotes a binding interface. It can be used instead of BindAddr
// if the interface is known but not the address. If both are provided, then
// Olric verifies that the interface has the bind address that is provided.
Interface string
// LogVerbosity denotes the level of message verbosity. The default value
// is 3. Valid values are between 1 to 6.
LogVerbosity int32
// Default LogLevel is DEBUG. Available levels: "DEBUG", "WARN", "ERROR", "INFO"
LogLevel string
// BindAddr denotes the address that Olric will bind to for communication
// with other Olric nodes.
BindAddr string
// BindPort denotes the address that Olric will bind to for communication
// with other Olric nodes.
BindPort int
// Client denotes configuration for TCP clients in Olric and the official
// Golang client.
Client *Client
// KeepAlivePeriod denotes whether the operating system should send
// keep-alive messages on the connection.
KeepAlivePeriod time.Duration
// IdleClose will automatically close idle connections after the specified duration.
// Use zero to disable this feature.
IdleClose time.Duration
// Timeout for bootstrap control
//
// An Olric node checks operation status before taking any action for the
// cluster events, responding incoming requests and running API functions.
// Bootstrapping status is one of the most important checkpoints for an
// "operable" Olric node. BootstrapTimeout sets a deadline to check
// bootstrapping status without blocking indefinitely.
BootstrapTimeout time.Duration
// Coordinator member pushes the routing table to cluster members in the case of
// node join or left events. It also pushes the table periodically. RoutingTablePushInterval
// is the interval between subsequent calls. Default is 1 minute.
RoutingTablePushInterval time.Duration
// TriggerBalancerInterval is interval between two sequential call of balancer worker.
TriggerBalancerInterval time.Duration
// The list of host:port which are used by memberlist for discovery.
// Don't confuse it with Name.
Peers []string
// PartitionCount is 271, by default.
PartitionCount uint64
// ReplicaCount is 1, by default.
ReplicaCount int
// Minimum number of successful reads to return a response for a read request.
ReadQuorum int
// Minimum number of successful writes to return a response for a write request.
WriteQuorum int
// Minimum number of members to form a cluster and run any query on the cluster.
MemberCountQuorum int32
// Switch to control read-repair algorithm which helps to reduce entropy.
ReadRepair bool
// Default value is SyncReplicationMode.
ReplicationMode int
// LoadFactor is used by consistent hashing function. It determines the maximum
// load for a server in the cluster. Keep it small.
LoadFactor float64
// Olric can send push cluster events to cluster.events channel. Available cluster events:
//
// * node-join-event
// * node-left-event
// * fragment-migration-event
// * fragment-received-event
//
// If you want to receive these events, set true to EnableClusterEventsChannel and subscribe to
// cluster.events channel. Default is false.
EnableClusterEventsChannel bool
// Default hasher is github.com/cespare/xxhash/v2
Hasher hasher.Hasher
// LogOutput is the writer where logs should be sent. If this is not
// set, logging will go to stderr by default. You cannot specify both LogOutput
// and Logger at the same time.
LogOutput io.Writer
// Logger is a custom logger which you provide. If Logger is set, it will use
// this for the internal logger. If Logger is not set, it will fall back to the
// behavior for using LogOutput. You cannot specify both LogOutput and Logger
// at the same time.
Logger *log.Logger
// DMaps denotes a global configuration for DMaps. You can still overwrite it
// by setting a DMap for a particular distributed map via DMaps.Custom field.
// Most of the fields are related with distributed cache implementation.
DMaps *DMaps
// JoinRetryInterval is the time gap between attempts to join an existing
// cluster.
JoinRetryInterval time.Duration
// MaxJoinAttempts denotes the maximum number of attempts to join an existing
// cluster before forming a new one.
MaxJoinAttempts int
// Callback function. Olric calls this after
// the server is ready to accept new connections.
Started func()
// ServiceDiscovery is a map that contains plugins implement ServiceDiscovery
// interface. See pkg/service_discovery/service_discovery.go for details.
ServiceDiscovery map[string]interface{}
// Interface denotes a binding interface. It can be used instead of
// memberlist.Loader.BindAddr if the interface is known but not the address.
// If both are provided, then Olric verifies that the interface has the bind
// address that is provided.
MemberlistInterface string
// Olric will broadcast a leave message but will not shut down the background
// listeners, meaning the node will continue participating in gossip and state
// updates.
//
// Sending a leave message will block until the leave message is successfully
// broadcast to a member of the cluster, if any exist or until a specified timeout
// is reached.
LeaveTimeout time.Duration
// MemberlistConfig is the memberlist configuration that Olric will
// use to do the underlying membership management and gossip. Some
// fields in the MemberlistConfig will be overwritten by Olric no
// matter what:
//
// * Name - This will always be set to the same as the NodeName
// in this configuration.
//
// * ClusterEvents - Olric uses a custom event delegate.
//
// * Delegate - Olric uses a custom delegate.
//
// You have to use NewMemberlistConfig to create a new one.
// Then, you may need to modify it to tune for your environment.
MemberlistConfig *memberlist.Config
}
Config is the configuration to create a Olric instance.
func New ¶
New returns a Config with sane defaults. If you change a configuration parameter, please run Sanitize and Validate functions respectively.
New takes an env parameter used by memberlist: local, lan and wan.
local:
DefaultLocalConfig works like DefaultConfig, however it returns a configuration that is optimized for a local loopback environments. The default configuration is still very conservative and errs on the side of caution.
lan:
DefaultLANConfig returns a sane set of configurations for Memberlist. It uses the hostname as the node name, and otherwise sets very conservative values that are sane for most LAN environments. The default configuration errs on the side of caution, choosing values that are optimized for higher convergence at the cost of higher bandwidth usage. Regardless, these values are a good starting point when getting started with memberlist.
wan:
DefaultWANConfig works like DefaultConfig, however it returns a configuration that is optimized for most WAN environments. The default configuration is still very conservative and errs on the side of caution.
func (*Config) Sanitize ¶
Sanitize sets default values to empty configuration variables, if it's possible.
func (*Config) SetupNetworkConfig ¶
SetupNetworkConfig tries to find an appropriate bindIP to bind and propagate.
type DMap ¶ added in v0.4.0
type DMap struct {
// Engine contains storage engine configuration and their implementations.
// If you don't have a custom storage engine implementation or configuration for
// the default one, just leave it empty.
Engine *Engine
// MaxIdleDuration denotes maximum time for each entry to stay idle in the
// DMap. It limits the lifetime of the entries relative to the time of the
// last read or write access performed on them. The entries whose idle period
// exceeds this limit are expired and evicted automatically. An entry is idle
// if no Get, GetEntry, Put, Expire on it. Configuration
// of MaxIdleDuration feature varies by preferred deployment method.
MaxIdleDuration time.Duration
// TTLDuration is useful to set a default TTL for every key/value pair a DMap
// instance.
TTLDuration time.Duration
// MaxKeys denotes maximum key count on a particular node. So if you have 10
// nodes with MaxKeys=100000, your key count in the cluster should be around
// MaxKeys*10=1000000
MaxKeys int
// MaxInuse denotes maximum amount of in-use memory on a particular node. So
// if you have 10 nodes with MaxInuse=100M (it has to be in bytes), amount of
// in-use memory should be around MaxInuse*10=1G
MaxInuse int
// LRUSamples denotes amount of randomly selected key count by the approximate
// LRU implementation. Lower values are better for high performance. It's 5
// by default.
LRUSamples int
// EvictionPolicy determines the eviction policy in use. It's NONE by default.
// Set as LRU to enable LRU eviction policy.
EvictionPolicy EvictionPolicy
}
DMap denotes configuration for a particular distributed map. Most of the fields are related with distributed cache implementation.
type DMaps ¶ added in v0.4.0
type DMaps struct {
// Engine contains configuration for a storage engine implementation. It may contain the implementation.
// See Engine itself.
Engine *Engine
// NumEvictionWorkers denotes the number of goroutines that are used to find
// keys for eviction. This is a global configuration variable. So you cannot set
// // different values per DMap.
NumEvictionWorkers int64
// MaxIdleDuration denotes maximum time for each entry to stay idle in the DMap.
// It limits the lifetime of the entries relative to the time of the last
// read or write access performed on them. The entries whose idle period exceeds
// this limit are expired and evicted automatically. An entry is idle if no Get,
// Put, Expire on it. Configuration of MaxIdleDuration feature varies by preferred
// deployment method.
MaxIdleDuration time.Duration
// TTLDuration is useful to set a default TTL for every key/value pair a
// distributed map instance.
TTLDuration time.Duration
// MaxKeys denotes maximum key count on a particular node. So if you have 10
// nodes with MaxKeys=100000, your key count in the cluster should be around
// MaxKeys*10=1000000
MaxKeys int
// MaxInuse denotes maximum amount of in-use memory on a particular node.
// So if you have 10 nodes with MaxInuse=100M (it has to be in bytes), amount
// of in-use memory should be around MaxInuse*10=1G
MaxInuse int
// LRUSamples denotes amount of randomly selected key count by the approximate
// LRU implementation. Lower values are better for high performance. It's
// 5 by default.
LRUSamples int
// EvictionPolicy determines the eviction policy in use. It's NONE by default.
// Set as LRU to enable LRU eviction policy.
EvictionPolicy EvictionPolicy
// CheckEmptyFragmentsInterval is the interval between two sequential calls of empty
// fragment cleaner. This is a global configuration variable. So you cannot set
// different values per DMap.
CheckEmptyFragmentsInterval time.Duration
// TriggerCompactionInterval is interval between two sequential call of compaction worker.
// This is a global configuration variable. So you cannot set
// different values per DMap.
TriggerCompactionInterval time.Duration
// Custom is useful to set custom cache config per DMap instance.
Custom map[string]DMap
}
DMaps denotes a global configuration for DMaps. You can still overwrite it by setting a DMap for a particular distributed map via Custom field. Most of the fields are related with distributed cache implementation.
type Engine ¶ added in v0.5.0
type Engine struct {
Name string
Implementation storage.Engine
// Config is a map that contains configuration of the storage engines, for
// both plugins and imported ones. If you want to use a storage engine other
// than the default one, you must set configuration for it.
Config map[string]interface{}
}
Engine contains storage engine configuration and their implementations. If you don't have a custom storage engine implementation or configuration for the default one, just call NewStorageEngine() function to use it with sane defaults.
func NewEngine ¶ added in v0.5.0
func NewEngine() *Engine
NewEngine initializes Engine with sane defaults. Olric will set its own storage engine implementation and related configuration, if there is no other engine.
type EvictionPolicy ¶
type EvictionPolicy string
EvictionPolicy denotes eviction policy. Currently: LRU or NONE.
type IConfig ¶ added in v0.4.0
type IConfig interface {
// Sanitize methods should be used to set defaults.
Sanitize() error
// Validate method should be used to find configuration errors.
Validate() error
}
IConfig is an interface that has to be implemented by Config and its nested structs. It provides a clear and granular way to sanitize and validate the configuration.
Source Files
¶
Directories
Click to show internal directories.
Click to hide internal directories.