README
¶
Pooly 
A package inspired by go-hostpool, to manage efficiently pools of connection across multiple hosts. Pooly uses multi-armed bandit algorithms for host selection in order to maximize service connectivity by avoiding faulty hosts.
Currently supported bandit strategies:
- Softmax
- Epsilon-greedy
- Round-robin
Basic Usage
Start two dummy netcat servers.
nc -l -p 1234 &
nc -l -p 12345 &
Get two connections from the "netcat" service using the NetDriver and the RoundRobin strategy (defaults).
package main
import "fmt"
import "github.com/3XX0/pooly"
func ping(s *pooly.Service) error {
c, err := s.GetConn()
if err != nil {
return err
}
defer c.Release(&err, pooly.HostUp)
_, err = c.NetConn().Write([]byte("ping\n"))
return err
}
func main() {
s := pooly.NewService("netcat", nil)
defer s.Close()
s.Add("127.0.0.1:1234")
s.Add("127.0.0.1:12345")
if err := ping(s); err != nil {
fmt.Println(err)
return
}
if err := ping(s); err != nil {
fmt.Println(err)
return
}
}
Metrics
Pooly exports useful service metrics provided that a statsd server is running and the option ServiceConfig.StatsdAddr is set accordingly.
The following metrics are available:
| metric | description |
|---|---|
| hosts.score | average score of the service hosts (in percentage) |
| hosts.count | number of hosts registered to the service |
| conns.count | number of connections spawned by the service |
| conns.fails | number of connection failures (temporary included) |
| conns.put.count | number of Release performed |
| conns.get.count | number of GetConn performed |
| conns.get.delay | average delay before receiving a connection from the service (in millisecond) |
| conns.get.fails | number of GetConn failures |
| conns.active.period | average time during which a connection was active (in millisecond) |
Example of a grafana dashboard using vizu
Simulations
In order to test the behavior of different strategies, Monte Carlo simulations may be run through the Go testing framework.
go test -tags 'montecarlo_simulation' -v pooly
Softmax strategy sample output
Epsilon-greedy strategy sample output
Documentation
¶
Index ¶
- Constants
- Variables
- func NewHTTPTransport(service *Service) *http.Transport
- func SetStatsdSampleRate(r float32)
- type Computer
- type Conn
- type Driver
- type EpsilonGreedy
- type Host
- type NetDriver
- func (n *NetDriver) Close(c *Conn)
- func (n *NetDriver) Dial(address string) (*Conn, error)
- func (n *NetDriver) SetConnTimeout(timeout time.Duration)
- func (n *NetDriver) SetReadTimeout(timeout time.Duration)
- func (n *NetDriver) SetWriteTimeout(timeout time.Duration)
- func (n *NetDriver) Temporary(err error) bool
- func (n *NetDriver) TestOnBorrow(c *Conn) error
- type Pool
- type PoolConfig
- type RoundRobin
- type Selecter
- type Service
- type ServiceConfig
- type SoftMax
Constants ¶
const ( DefaultMaxConns = 30 DefaultConnRetries = 3 DefaultRetryDelay = 10 * time.Millisecond DefaultPrespawnConns = 1 DefaultGetAttempts = 3 DefaultCloseDeadline = 30 * time.Second DefaultDecayDuration = 1 * time.Minute DefaultMemoizeScoreDuration = 100 * time.Millisecond )
Pooly default constants.
const ( HostDown float64 = 0 HostUp = 1 )
Predefined scores (all or nothing).
Variables ¶
var ( ErrInvalidArg = errors.New("pooly: invalid argument") ErrPoolClosed = errors.New("pooly: pool is closed") ErrOpTimeout = errors.New("pooly: operation timed out") ErrNoHostAvailable = errors.New("pooly: no host available") )
Pooly global errors.
Functions ¶
func NewHTTPTransport ¶
func SetStatsdSampleRate ¶
func SetStatsdSampleRate(r float32)
SetStatsdSampleRate sets the metrics sampling rate. It takes a float between 0 and 1 which defines the percentage of the time where pooly will send its service metrics to statsd.
Types ¶
type Computer ¶
Computer describes the interface responsible of computing the resulting score of a host. It takes the initial score and returns one computed using a predefined function (e.g exp, log ...)
type Conn ¶
type Conn struct {
// contains filtered or unexported fields
}
Conn abstracts user connections that are part of a Pool.
func NewConn ¶
func NewConn(i interface{}) *Conn
NewConn creates a new connection container, wrapping up a user defined connection object.
func (*Conn) Interface ¶
func (c *Conn) Interface() interface{}
Interface returns an interface referring to the underlying user object.
func (*Conn) NetConn ¶
NetConn is a helper for underlying user objects that satisfy the standard library net.Conn interface.
func (*Conn) Release ¶
Release releases the connection back to its linked service. It takes an error state which defines whether or not the connection failed during operation and a score between 0 and 1 which describes how well the connection performed (e.g inverse response time, up/down ...). If the error state indicates a fatal error (determined by Driver.Temporary), the score is forced to the value 0 (HostDown).
type Driver ¶
type Driver interface {
// Dial is a function that given an address, establishes a connection with a remote host.
// It returns the connection created or an error on failure.
Dial(string) (*Conn, error)
// Close closes the given connection.
Close(*Conn)
// TestOnBorrow is a function that, given a connection, tests it and returns an error on failure.
TestOnBorrow(*Conn) error
// Temporary determines whether the error is temporary or fatal for the connection.
// On fatal error, the connection will be garbage collected.
Temporary(error) bool
}
Driver describes the interface responsible of creating/deleting/testing pool connections.
type EpsilonGreedy ¶
type EpsilonGreedy struct {
// contains filtered or unexported fields
}
EpsilonGreedy strategy selects generally the host having the highest score (greedy) but every once in a while it will randomly explore for other alternatives. The epsilon parameter (0-1) defines the proportion that the exploration phase occupies (e.g 1 for 100%).
func NewEpsilonGreedy ¶
func NewEpsilonGreedy(epsilon float32) *EpsilonGreedy
NewEpsilonGreedy creates a new EpsilonGreedy bandit strategy.
type NetDriver ¶
type NetDriver struct {
// contains filtered or unexported fields
}
NetDriver is a predefined driver for handling standard net.Conn objects.
func NewNetDriver ¶
NewNetDriver instantiates a new NetDriver, ready to be used in a PoolConfig.
func (*NetDriver) SetConnTimeout ¶
SetConnTimeout sets the dialing timeout on a net.Conn object.
func (*NetDriver) SetReadTimeout ¶
SetReadTimeout sets the read timeout on a net.Conn object.
func (*NetDriver) SetWriteTimeout ¶
SetWriteTimeout sets the write timeout on a net.Conn object.
func (*NetDriver) TestOnBorrow ¶
TestOnBorrow does nothing.
type Pool ¶
type Pool struct {
*PoolConfig
// contains filtered or unexported fields
}
Pool maintains a pool of connections. The application calls the Get method to get a connection from the pool and the Put method to return the connection to the pool. New can be called to allocate more connections in the background. When one is done with the pool, Close will cleanup all the connections resources. The pool itself will adapt to the demand by spawning and destroying connections as needed. In order to tweak its behavior, settings like ConnIdleTimeout and MaxConns may be used.
func NewPool ¶
func NewPool(address string, c *PoolConfig) *Pool
NewPool creates a new pool of connections. If no configuration is specified (nil), defaults values are used.
func (*Pool) ActiveConns ¶
ActiveConns returns the number of connections handled by the pool thus far.
func (*Pool) Close ¶
Close closes the pool, thus destroying all connections. It returns when all spawned connections have been successfully garbage collected. After a successful call to Close, the pool can not be used again.
func (*Pool) ForceClose ¶
ForceClose forces the termination of an ongoing Close operation. It returns true if Close is interrupted successfully, false otherwise. Note that all pending connections unacknowledged by Close will be left unchanged and won't ever be destroyed.
type PoolConfig ¶
type PoolConfig struct {
// Connection driver (TCP NetDriver by default).
Driver Driver
// Close connections after remaining idle for this duration.
// If the value is zero (default), then idle connections are not closed.
ConnIdleTimeout time.Duration
// Defines the duration during which Get operations will try to return a connection from the pool.
// If the value is zero (default), then Get should wait forever.
WaitTimeout time.Duration
// Maximum number of connections allowed in the pool (DefaultMaxConns by default).
MaxConns int32
// Number of connection retry (DefaultConnRetries by default).
ConnRetries int
// Time interval between connection retry (DefaultRetryDelay by default).
RetryDelay time.Duration
}
PoolConfig defines the pool configuration options.
type RoundRobin ¶
RoundRobin strategy selects hosts in circular manner with every request returning the next host in line.
func NewRoundRobin ¶
func NewRoundRobin() *RoundRobin
NewRoundRobin creates a new RoundRobin bandit strategy.
type Selecter ¶
Selecter describes the interface responsible of selecting a host among the ones registered in the service.
type Service ¶
type Service struct {
*ServiceConfig
sync.RWMutex
// contains filtered or unexported fields
}
Service manages several hosts, every one of them having a connection pool (see Pool). It computes periodically hosts scores and learns about the best alternatives according to the BanditStrategy option. Hosts are added or removed from the service via Add and Remove respectively. The application calls the GetConn method to get a connection and releases it through the Conn.Release interface. When one is done with the pool, Close will cleanup all the service resources.
func NewService ¶
func NewService(name string, c *ServiceConfig) (*Service, error)
NewService creates a new service given a unique name. If no configuration is specified (nil), defaults values are used.
func (*Service) Add ¶
Add adds a given host to the service. The effect of such operation may not be reflected immediately.
func (*Service) Close ¶
func (s *Service) Close()
Close closes the service, thus destroying all hosts and their respective pool. After a call to Close, the service can not be used again.
func (*Service) GetConn ¶
GetConn returns a connection from the service. The host serving the connection is chosen according to the BanditStrategy policy in place.
type ServiceConfig ¶
type ServiceConfig struct {
PoolConfig
// Number of connections to prespawn on hosts additions (DefaultPrespawnConns by default).
PrespawnConns uint
// Number of attempts to get a connection from the service before giving up (DefaultGetAttempts by default).
GetAttempts uint
// Deadline after which pools are forced closed (see Pool.ForceClose) (DefaultCloseDeadline by default).
CloseDeadline time.Duration
// Defines the time interval taken into account when scores are computed (DefaultDecayDuration by default).
// Scores are calculated using a weighted average over the course of this duration (recent feedbacks get higher weight).
DecayDuration time.Duration
// Time interval between two successive hosts scores computations.
// Each score is calculated and cached for this duration (DefaultMemoizeScoreDuration by default).
MemoizeScoreDuration time.Duration
// Optional score calculator (none by default).
ScoreCalculator Computer
// Multi-armed bandit strategy used for host selection (RoundRobin by default).
// The tradeoff faced by the service at each GetConn is between "exploitation" (choose hosts having the highest score)
// and "exploration" (find about the expected score of other hosts).
// The key here is to find the right balance between "exploration" and "exploitation" of hosts given their respective score.
// Some strategies will favor fairness while others will prefer to pick hosts based on how well they perform.
BanditStrategy Selecter
// Address and port of a statsd server to collect and aggregate pooly service metrics (none by default).
StatsdAddr string
}
ServiceConfig defines the service configuration options.
type SoftMax ¶
type SoftMax struct {
// contains filtered or unexported fields
}
SoftMax strategy varies host selection probabilities as a graded function of their estimated scores. The temperature parameter is used to tweak the algorithm behavior: high temperature (+inf) means that all hosts will have nearly the same probability of being selected (equiprobable) low temperature (+0) favors a greedy selection and will tend to select hosts having the highest scores
func NewSoftMax ¶
NewSoftMax creates a new SoftMax bandit strategy.
Source Files
Directories