Documentation
¶
Index ¶
Constants ¶
View Source
const ( DefaultDialTimeout = 5 * time.Second DefaultReadTimeout = 3 * time.Second DefaultWriteTimeout = 3 * time.Second DefaultPoolTimeout = 3 * time.Second )
View Source
const ( // SyncReplicationMode enables sync replication mode which means that the // caller is blocked until write/delete operation is applied by replica // owners. The default mode is SyncReplicationMode SyncReplicationMode = 0 // AsyncReplicationMode enables async replication mode which means that // write/delete operations are done in a background task. AsyncReplicationMode = 1 )
View Source
const ( LogLevelDebug = "DEBUG" LogLevelWarn = "WARN" LogLevelError = "ERROR" LogLevelInfo = "INFO" )
View Source
const ( // DefaultPort is for Olric DefaultPort = 3320 // DefaultDiscoveryPort is for memberlist DefaultDiscoveryPort = 3322 // DefaultPartitionCount denotes default partition count in the cluster. DefaultPartitionCount = 271 // DefaultLoadFactor is used by the consistent hashing function. Keep it small. DefaultLoadFactor = 1.25 // DefaultLogLevel determines the log level without extra configuration. // It's DEBUG. DefaultLogLevel = LogLevelDebug // DefaultLogVerbosity denotes default log verbosity level. // // * flog.V(1) - Generally useful for this to ALWAYS be visible to an operator // * Programmer errors // * Logging extra info about a panic // * CLI argument handling // * flog.V(2) - A reasonable default log level if you don't want verbosity. // * Information about config (listening on X, watching Y) // * Errors that repeat frequently that relate to conditions that can be // corrected (pod detected as unhealthy) // * flog.V(3) - Useful steady state information about the service and // important log messages that may correlate to // significant changes in the system. This is the recommended default log // level for most systems. // * Logging HTTP requests and their exit code // * System state changing (killing pod) // * Controller state change events (starting pods) // * Scheduler log messages // * flog.V(4) - Extended information about changes // * More info about system state changes // * flog.V(5) - Debug level verbosity // * Logging in particularly thorny parts of code where you may want to come // back later and check it // * flog.V(6) - Trace level verbosity // * Context to understand the steps leading up to neterrors and warnings // * More information for troubleshooting reported issues DefaultLogVerbosity = 3 // MinimumReplicaCount denotes default and minimum replica count in an Olric // cluster. MinimumReplicaCount = 1 // DefaultBootstrapTimeout denotes default timeout value to check bootstrapping // status. DefaultBootstrapTimeout = 10 * time.Second // DefaultJoinRetryInterval denotes a time gap between sequential join attempts. DefaultJoinRetryInterval = time.Second // DefaultLeaveTimeout is the default value of maximum amount of time before DefaultLeaveTimeout = 5 * time.Second // DefaultMaxJoinAttempts denotes a maximum number of failed join attempts // before forming a standalone cluster. DefaultMaxJoinAttempts = 10 // MinimumMemberCountQuorum denotes minimum required count of members to form // a cluster. MinimumMemberCountQuorum = 1 // DefaultLRUSamples is a sane default for randomly selected keys // in approximate LRU implementation. It's 5. DefaultLRUSamples int = 5 // LRUEviction assigns this as EvictionPolicy in order to enable LRU eviction // algorithm. LRUEviction EvictionPolicy = "LRU" // DefaultStorageEngine denotes the storage engine implementation provided by // Olric project. DefaultStorageEngine = "kvstore" // DefaultRoutingTablePushInterval is interval between routing table push events. DefaultRoutingTablePushInterval = time.Minute // DefaultCheckEmptyFragmentsInterval is the default value of interval between // two sequential call of empty fragment cleaner. DefaultCheckEmptyFragmentsInterval = time.Minute // DefaultKeepAlivePeriod is the default value of TCP keepalive. It's 300 seconds. // This option is useful in order to detect dead peers (clients that cannot // be reached even if they look connected). Moreover, if there is network // equipment between clients and servers that need to see some traffic in // order to take the connection open, the option will prevent unexpected // connection closed events. DefaultKeepAlivePeriod = 300 * time.Second )
Variables ¶
This section is empty.
Functions ¶
func NewMemberlistConfig ¶
func NewMemberlistConfig(env string) (*memberlist.Config, error)
NewMemberlistConfig returns a new memberlist.Config for a given environment.
It takes an env parameter: local, lan and wan.
local: DefaultLocalConfig works like DefaultConfig, however it returns a configuration that is optimized for a local loopback environments. The default configuration is still very conservative and errs on the side of caution.
lan: DefaultLANConfig returns a sane set of configurations for Memberlist. It uses the hostname as the node name, and otherwise sets very conservative values that are sane for most LAN environments. The default configuration errs on the side of caution, choosing values that are optimized for higher convergence at the cost of higher bandwidth usage. Regardless, these values are a good starting point when getting started with memberlist.
wan: DefaultWANConfig works like DefaultConfig, however it returns a configuration that is optimized for most WAN environments. The default configuration is still very conservative and errs on the side of caution.
Types ¶
type Client ¶ added in v0.3.0
type Client struct {
// Timeout for TCP dial.
//
// The timeout includes name resolution, if required. When using TCP, and the host in the address parameter
// resolves to multiple IP addresses, the timeout is spread over each consecutive dial, such that each is
// given an appropriate fraction of the time to connect.
DialTimeout time.Duration
// Timeout for socket reads. If reached, commands will fail
// with a timeout instead of blocking. Use value -1 for no timeout and 0 for default.
// Default is DefaultReadTimeout
ReadTimeout time.Duration
// Timeout for socket writes. If reached, commands will fail
// with a timeout instead of blocking.
// Default is DefaultWriteTimeout
WriteTimeout time.Duration
// KeepAlive specifies the interval between keep-alive
// probes for an active network connection.
// If zero, keep-alive probes are sent with a default value
// (currently 15 seconds), if supported by the protocol and operating
// system. Network protocols or operating systems that do
// not support keep-alives ignore this field.
// If negative, keep-alive probes are disabled.
KeepAlive time.Duration
// Minimum TCP connection count in the pool for a host:port
MinConn int
// Maximum TCP connection count in the pool for a host:port
MaxConn int
// Timeout for getting a new connection from the pool. If reached, commands will fail
// with a timeout instead of blocking. Use value -1 for no timeout and 0 for default.
// Default is DefaultPoolTimeout
PoolTimeout time.Duration
}
Client denotes configuration for TCP clients in Olric and the official Golang client.
func NewClient ¶ added in v0.3.0
func NewClient() *Client
NewClient returns a new configuration object for clients.
func (*Client) HasTimeout ¶ added in v0.3.0
type Config ¶
type Config struct {
// Interface denotes a binding interface. It can be used instead of BindAddr
// if the interface is known but not the address. If both are provided, then
// Olric verifies that the interface has the bind address that is provided.
Interface string
// LogVerbosity denotes the level of message verbosity. The default value
// is 3. Valid values are between 1 to 6.
LogVerbosity int32
// Default LogLevel is DEBUG. Available levels: "DEBUG", "WARN", "ERROR", "INFO"
LogLevel string
// BindAddr denotes the address that Olric will bind to for communication
// with other Olric nodes.
BindAddr string
// BindPort denotes the address that Olric will bind to for communication
// with other Olric nodes.
BindPort int
// Client denotes configuration for TCP clients in Olric and the official
// Golang client.
Client *Client
// KeepAlivePeriod denotes whether the operating system should send
// keep-alive messages on the connection. This option is useful in order to
// detect dead peers (clients that cannot be reached even if they look
// connected). Moreover, if there is network equipment between clients and
// servers that need to see some traffic in order to take the connection open,
// the option will prevent unexpected connection closed events.
KeepAlivePeriod time.Duration
// Timeout for bootstrap control
//
// An Olric node checks operation status before taking any action for the
// cluster events, responding incoming requests and running API functions.
// Bootstrapping status is one of the most important checkpoints for an
// "operable" Olric node. BootstrapTimeout sets a deadline to check
// bootstrapping status without blocking indefinitely.
BootstrapTimeout time.Duration
// RoutingTablePushInterval is interval between routing table push events.
RoutingTablePushInterval time.Duration
// The list of host:port which are used by memberlist for discovery.
// Don't confuse it with Name.
Peers []string
// PartitionCount is 271, by default.
PartitionCount uint64
// ReplicaCount is 1, by default.
ReplicaCount int
// Minimum number of successful reads to return a response for a read request.
ReadQuorum int
// Minimum number of successful writes to return a response for a write request.
WriteQuorum int
// Minimum number of members to form a cluster and run any query on the cluster.
MemberCountQuorum int32
// Switch to control read-repair algorithm which helps to reduce entropy.
ReadRepair bool
// Default value is SyncReplicationMode.
ReplicationMode int
// LoadFactor is used by consistent hashing function. It determines the maximum
// load for a server in the cluster. Keep it small.
LoadFactor float64
// Default hasher is github.com/cespare/xxhash
Hasher hasher.Hasher
// Default Serializer implementation uses gob for encoding/decoding.
Serializer serializer.Serializer
// LogOutput is the writer where logs should be sent. If this is not
// set, logging will go to stderr by default. You cannot specify both LogOutput
// and Logger at the same time.
LogOutput io.Writer
// Logger is a custom logger which you provide. If Logger is set, it will use
// this for the internal logger. If Logger is not set, it will fall back to the
// behavior for using LogOutput. You cannot specify both LogOutput and Logger
// at the same time.
Logger *log.Logger
// DMaps denotes a global configuration for DMaps. You can still overwrite it
// by setting a DMap for a particular distributed map via DMaps.Custom field.
// Most of the fields are related with distributed cache implementation.
DMaps *DMaps
// JoinRetryInterval is the time gap between attempts to join an existing
// cluster.
JoinRetryInterval time.Duration
// MaxJoinAttempts denotes the maximum number of attemps to join an existing
// cluster before forming a new one.
MaxJoinAttempts int
// Callback function. Olric calls this after
// the server is ready to accept new connections.
Started func()
// ServiceDiscovery is a map that contains plugins implement ServiceDiscovery
// interface. See pkg/service_discovery/service_discovery.go for details.
ServiceDiscovery map[string]interface{}
// Interface denotes a binding interface. It can be used instead of
// memberlist.Loader.BindAddr if the interface is known but not the address.
// If both are provided, then Olric verifies that the interface has the bind
// address that is provided.
MemberlistInterface string
// Olric will broadcast a leave message but will not shut down the background
// listeners, meaning the node will continue participating in gossip and state
// updates.
//
// Sending a leave message will block until the leave message is successfully
// broadcast to a member of the cluster, if any exist or until a specified timeout
// is reached.
LeaveTimeout time.Duration
// MemberlistConfig is the memberlist configuration that Olric will
// use to do the underlying membership management and gossip. Some
// fields in the MemberlistConfig will be overwritten by Olric no
// matter what:
//
// * Name - This will always be set to the same as the NodeName
// in this configuration.
//
// * ClusterEvents - Olric uses a custom event delegate.
//
// * Delegate - Olric uses a custom delegate.
//
// You have to use NewMemberlistConfig to create a new one.
// Then, you may need to modify it to tune for your environment.
MemberlistConfig *memberlist.Config
// StorageEngines contains storage engine configuration and their implementations.
StorageEngines *StorageEngines
}
Config is the configuration to create a Olric instance.
func New ¶
New returns a Config with sane defaults. It takes an env parameter used by memberlist: local, lan and wan.
local:
DefaultLocalConfig works like DefaultConfig, however it returns a configuration that is optimized for a local loopback environments. The default configuration is still very conservative and errs on the side of caution.
lan:
DefaultLANConfig returns a sane set of configurations for Memberlist. It uses the hostname as the node name, and otherwise sets very conservative values that are sane for most LAN environments. The default configuration errs on the side of caution, choosing values that are optimized for higher convergence at the cost of higher bandwidth usage. Regardless, these values are a good starting point when getting started with memberlist.
wan:
DefaultWANConfig works like DefaultConfig, however it returns a configuration that is optimized for most WAN environments. The default configuration is still very conservative and errs on the side of caution.
func (*Config) Sanitize ¶
Sanitize sets default values to empty configuration variables, if it's possible.
func (*Config) SetupNetworkConfig ¶
SetupNetworkConfig tries to find an appropriate bindIP to bind and propagate.
type DMap ¶ added in v0.4.0
type DMap struct {
// MaxIdleDuration denotes maximum time for each entry to stay idle in the
// DMap. It limits the lifetime of the entries relative to the time of the
// last read or write access performed on them. The entries whose idle period
// exceeds this limit are expired and evicted automatically. An entry is idle
// if no Get, GetEntry, Put, PutEx, Expire, PutIf, PutIfEx on it. Configuration
// of MaxIdleDuration feature varies by preferred deployment method.
MaxIdleDuration time.Duration
// TTLDuration is useful to set a default TTL for every key/value pair a DMap
// instance.
TTLDuration time.Duration
// MaxKeys denotes maximum key count on a particular node. So if you have 10
// nodes with MaxKeys=100000, your key count in the cluster should be around
// MaxKeys*10=1000000
MaxKeys int
// MaxInuse denotes maximum amount of in-use memory on a particular node. So
// if you have 10 nodes with MaxInuse=100M (it has to be in bytes), amount of
// in-use memory should be around MaxInuse*10=1G
MaxInuse int
// LRUSamples denotes amount of randomly selected key count by the approximate
// LRU implementation. Lower values are better for high performance. It's 5
// by default.
LRUSamples int
// EvictionPolicy determines the eviction policy in use. It's NONE by default.
// Set as LRU to enable LRU eviction policy.
EvictionPolicy EvictionPolicy
// Name of the storage engine. The default one is kvstore. Leave it empty if
// you want to use the default one.
StorageEngine string
}
DMap denotes configuration for a particular distributed map. Most of the fields are related with distributed cache implementation.
type DMaps ¶ added in v0.4.0
type DMaps struct {
// NumEvictionWorkers denotes the number of goroutines that's used to find
// keys for eviction.
NumEvictionWorkers int64
// MaxIdleDuration denotes maximum time for each entry to stay idle in the DMap.
// It limits the lifetime of the entries relative to the time of the last
// read or write access performed on them. The entries whose idle period exceeds
// this limit are expired and evicted automatically. An entry is idle if no Get,
// GetEntry, Put, PutEx, Expire, PutIf, PutIfEx on it. Configuration of
// MaxIdleDuration feature varies by preferred deployment method.
MaxIdleDuration time.Duration
// TTLDuration is useful to set a default TTL for every key/value pair a
// distributed map instance.
TTLDuration time.Duration
// MaxKeys denotes maximum key count on a particular node. So if you have 10
// nodes with MaxKeys=100000, your key count in the cluster should be around
// MaxKeys*10=1000000
MaxKeys int
// MaxInuse denotes maximum amount of in-use memory on a particular node.
// So if you have 10 nodes with MaxInuse=100M (it has to be in bytes), amount
// of in-use memory should be around MaxInuse*10=1G
MaxInuse int
// LRUSamples denotes amount of randomly selected key count by the approximate
// LRU implementation. Lower values are better for high performance. It's
// 5 by default.
LRUSamples int
// EvictionPolicy determines the eviction policy in use. It's NONE by default.
// Set as LRU to enable LRU eviction policy.
EvictionPolicy EvictionPolicy
// Name of the storage engine. The default one is kvstore. Leave it empty if
// you want to use the default one.
StorageEngine string
// CheckEmptyFragmentsInterval is interval between two sequential call of empty
// fragment cleaner.
CheckEmptyFragmentsInterval time.Duration
// Custom is useful to set custom cache config per DMap instance.
Custom map[string]DMap
}
DMaps denotes a global configuration for DMaps. You can still overwrite it by setting a DMap for a particular distributed map via Custom field. Most of the fields are related with distributed cache implementation.
type EvictionPolicy ¶
type EvictionPolicy string
EvictionPolicy denotes eviction policy. Currently: LRU or NONE.
type IConfig ¶ added in v0.4.0
type IConfig interface {
// Sanitize methods should be used to set defaults.
Sanitize() error
// Validate method should be used to find configuration errors.
Validate() error
}
IConfig is an interface that has to be implemented by Config and its nested structs. It provides a clear and granular way to sanitize and validate the configuration.
type StorageEngines ¶ added in v0.4.0
type StorageEngines struct {
// Plugins is an array that contains the paths of storage engine plugins.
// These plugins have to implement storage.Engine interface.
Plugins []string
// Impls is a map that contains storage engines with their names.
// This is useful to import and use external storage engine implementations.
Impls map[string]storage.Engine
// Config is a map that contains configuration of the storage engines, for
// both plugins and imported ones. If you want to use a storage engine other
// than the default one, you must set configuration for it.
Config map[string]map[string]interface{}
}
StorageEngines contains storage engine configuration and their implementations. If you don't have a custom storage engine implementation or configuration for the default one, just call NewStorageEngine() function to use it with sane defaults.
func NewStorageEngine ¶ added in v0.4.0
func NewStorageEngine() *StorageEngines
NewStorageEngine initializes StorageEngine configuration with sane defaults. Olric will set its own storage engine implementation and related configuration, if there is no other engine.
func (*StorageEngines) Sanitize ¶ added in v0.4.0
func (s *StorageEngines) Sanitize() error
Sanitize sets default values to empty configuration variables, if it's possible.
func (*StorageEngines) Validate ¶ added in v0.4.0
func (s *StorageEngines) Validate() error
Validate finds errors in the current configuration.
Source Files
¶
Directories
Click to show internal directories.
Click to hide internal directories.