serversets

README

go.serversets

Package go.serversets provides an simple interface for service discovery using Apache Zookeeper. Servers/endpoints register themselves and clients always have an updated host list.

This core package just provides a list of hostnames and ports. Sub-packages wrap the endpoint list for common use cases:

• mcset provides consistent hashing over a set of memcache hosts.
• httpset round-robins standard HTTP requests to the set of hosts.
• fixedset severset watch without the zookeeper. Take advantage of
• thriftset does "least request" load balancing around the given endpoints.

This package is used internally at Strava for Finagle service discovery and memcache node registration.

Usage

First, create a ServerSet defining an environment and service name.

zookeepers := []string{"zk01.internal", "zk02.internal", "zk03.internal"}
serverSet := serversets.New(serversets.Staging, "service_name", zookeepers)

This doesn't actually connect to the Zookeeper servers, just defines the namespace. Methods like RegisterEndpoint and Watch will return an error if they can't connect. Available environment constants: serversets.Local, serversets.Staging, serversets.Production and serversets.Test.

Register an Endpoint

Service endpoints/producers/servers should register themselves as an endpoint. Example:

pingFunction := func() error {
	return nil
}

endpoint, err := serverSet.RegisterEndpoint(
	localIP,
	servicePort,
	pingFunction)

The ping function can be nil. But if it's not, it'll be checked every second, by default. If there is an error the endpoint will be unregistered. Once the issue is resolved it'll be reregistered automatically. This allows for registering external processes that may fail independently of the monitoring process.

Watch the list of available endpoints, for consumers

watch, err = serverSet.Watch()
if err != nil {
	// probably something wrong with connecting to Zookeeper
	panic(err)
}

endpoints := watch.Endpoints()
for {
	<-watch.Event()
	// endpoint list changed
}

The watch.Event() channel will be triggered whenever the endpoint list changes and watch.Endpoints() will contain the updated list of available endpoints.

Finagle Compatibility

The Zookeeper zNode data is designed to be compatible with Finagle ServerSets. It is just a matter or matching the namespaces in the ServerSet declarations. This library registers endpoints to zNodes similar to /discovery/staging/service_name/member_0000000318.

A Scala snippet to register an endpoint discoverable by the watch created above:

val serverHost: java.net.InetSocketAddress
val zookeeperHost: java.net.InetSocketAddress

val zookeeperClient = new ZookeeperClient(sessionTimeout, zookeeperHost)
val serverSet = new ServerSetImpl(zk.zookeeperClient, "/discovery/staging/service_name")
val cluster = new ZookeeperServerSetCluster(serverSet)

cluster.join(serverHost)

The namespaces used by this library are completely configurable. One just needs to defining their own BaseZnodePath function.

Dependencies

Dependencies are vendored in the /vendor directory. This is compatible with the 'Go 1.5 vendor experiment.' Set GO15VENDOREXPERIMENT=1 during the build to make use of these packages.

Tests

Tests require a Zookeeper server. The default is "localhost" but a different host can be used by changing the TestServer variable in serverset_test.go

go test github.com/dongnguyenvt/go.serversets/...

Potential Improvements and Contributing

This library simply provides a list of active endpoints. But it would nice if it did some load balancing, error checking, retries etc. Simple versions of this are available for memcache and http but they can be improved. So, if you have some ideas, submit a pull request.

Documentation

Index

• Constants
• Variables
• type Endpoint
  • func (ep *Endpoint) Close()
• type Environment
• type ServerSet
  • func New(environment Environment, service string, zookeepers []string) *ServerSet
  • func (ss *ServerSet) RegisterEndpoint(host string, port int, ping func() error) (*Endpoint, error)
  • func (ss *ServerSet) Watch() (*Watch, error)
  • func (ss *ServerSet) ZookeeperServers() []string
• type Watch
  • func (w *Watch) Close()
  • func (w *Watch) Endpoints() []string
  • func (w *Watch) Event() <-chan struct{}
  • func (w *Watch) IsClosed() bool

Constants

const (
	// SOH control character
	SOH = "\x01"
)

Variables

var (
	// BaseDirectory is the Zookeeper namespace that all nodes made by this package will live.
	// This path must begin with '/'
	BaseDirectory = "/discovery"

	// MemberPrefix is prefix for the Zookeeper sequential ephemeral nodes.
	// member_ is used by Finagle server sets.
	MemberPrefix = "member_"
)

var BaseZnodePath = func(environment Environment, service string) string {
	return BaseDirectory + "/" + string(environment) + "/" + service
}

BaseZnodePath allows for a custom Zookeeper directory structure. This function should return the path where you want the service's members to live. Default is `BaseDirectory + "/" + environment + "/" + service` where the default base directory is `/discovery`

var DefaultZKTimeout = 5 * time.Second

DefaultZKTimeout is the zookeeper timeout used if it is not overwritten.

Types

type Endpoint

type Endpoint struct {
	*ServerSet
	PingRate   time.Duration // default/initial is 1 second
	CloseEvent chan struct{}
	// contains filtered or unexported fields
}

An Endpoint is a service (host and port) registered on Zookeeper to be discovered by clients/watchers.

func (*Endpoint) Close

func (ep *Endpoint) Close()

Close blocks until the client connection to Zookeeper is closed. If already called, will simply return, even if in the process of closing.

type Environment

type Environment string

An Environment is the test/staging/production state of the service.

const (
	Local      Environment = "local"
	Production Environment = "prod"
	Test       Environment = "test"
	Staging    Environment = "staging"
)

Typically used environments

type ServerSet

type ServerSet struct {
	ZKTimeout time.Duration
	// contains filtered or unexported fields
}

A ServerSet represents a service with a set of servers that may change over time. The master lists of servers is kept as ephemeral nodes in Zookeeper.

func New

func New(environment Environment, service string, zookeepers []string) *ServerSet

New creates a new ServerSet object that can then be watched or have an endpoint added to. The service name must not contain any slashes. Will panic if it does.

func (*ServerSet) RegisterEndpoint

func (ss *ServerSet) RegisterEndpoint(host string, port int, ping func() error) (*Endpoint, error)

RegisterEndpoint registers a host and port as alive. It creates the appropriate Zookeeper nodes and watchers will be notified this server/endpoint is available.

func (*ServerSet) Watch

func (ss *ServerSet) Watch() (*Watch, error)

Watch creates a new watch on this server set. Changes to the set will update watch.Endpoints() and an event will be sent to watch.Event right after that happens.

func (*ServerSet) ZookeeperServers

func (ss *ServerSet) ZookeeperServers() []string

ZookeeperServers returns the Zookeeper servers this set is using. Useful to check if everything is configured correctly.

type Watch

type Watch struct {
	LastEvent  time.Time
	EventCount int
	// contains filtered or unexported fields
}

A Watch keeps tabs on a server set in Zookeeper and notifies via the Event() channel when the list of servers changes. The list of servers is updated automatically and will be up to date when the Event is sent.

func (*Watch) Close

func (w *Watch) Close()

Close blocks until the underlying Zookeeper connection is closed.

func (*Watch) Endpoints

func (w *Watch) Endpoints() []string

Endpoints returns a slice of the current list of servers/endpoints associated with this watch.

func (*Watch) Event

func (w *Watch) Event() <-chan struct{}

Event returns the event channel. This channel will get an object whenever something changes with the list of endpoints.

func (*Watch) IsClosed

func (w *Watch) IsClosed() bool

IsClosed returns if this watch has been closed. This is a way for libraries wrapping this package to know if their underlying watch is closed and should stop looking for events.