etcd_recipes

package
v0.0.0-...-f310e5b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 12, 2016 License: Apache-2.0 Imports: 10 Imported by: 0

README

etcd-connector

A simple wrapper for the etcd client that also provides a set of common distributed coordination recipes.

Introduction

etcd-connector aims to provide the user a set of recipes that are commonly used in a distributed systems project. The etcd-connector wraps the etcd-client which is used to talk to the etcd service.

etcd is an open source distributed key-value store that is based on Raft consensus protocol. etcd is typically used to store cluster wide configuration information and to perform cluster wide coordination operations. The project is implemented & maintained by CoreOS and more information about the project can be found here.

Implementation Details

Currently the etcd-connector has been implemented in golang. Although similar schemes can be used and implemented in other programming languages.

As mentioned earlier the etcd-connector wraps the etcd-client, that comes with etcd, by including it as an anonymous field. This enables the user to perform all the etcd-client operations using the etcd-connector instance. It also exposes a set of new methods that are used to instantiate the following recipes:

The etcd-connector works with v2 etcd APIs. It has been tested against etcd version 2.2.1

Getting etcd-connector

As the etcd-connector uses etcd-client the user will have to execute the following commands to get access to the code:

go get github.com/coreos/etcd/client
go get github.com/asyalabs/etcd-connector

Once the etcd-connector code base if fetched the consumer can instantiate the connector in the following way:

import etcdc "github.com/asyalabs/etcd-connector/Go"

server_list := etcdc.PrepareEndpointList([]string{"host1", "hosts2", "host3"}, 4001)
conn, _ := etcdc.NewEtcdConnector(server_list, "my_namespace")

The parameter "my_namespace" is essentially a directory under etcd root ("/"). This way applications can define their namespace and operate within that thereby enabling multiple applications to use the same etcd service.

Recipes

The following sections provide a brief overview of all the recipes.

Leader Elector

A recipe to elect a leader amongst a set of participants. All the participants race to create a key at an agreed upon path in the etcd namespace. Only one of them will succeed and assume the position of the leader. The leader will then renew the TTL on the key to stay elected. The other participants wait for the current leader to give up. When the leader dies the TTL on the key expires and the followers will try to take the leader role.

The following sample snippet describes the usage:

ttl := 10 * time.Second

// Construct the key path within the namespace.
keyPath := conn.ConstructPath("elections", "service_name")

// Instantiate a leader elector recipe instance.
ldrElect := conn.NewLeaderElector(keyPath, "participant identity", ttl)

// Start participating.
statusCh, _ := ldrElect.Start()

// Listen to the election response.
go func() {
	for eleResp := range statusCh {
		if eleResp.Status == etcdc.LEStatusLeader {
			// Perform leader activities.
		} else if eleResp.Status == etcdc.LEStatusFollower {
			// Perform follower activities.
		}
	}
}()
...
...
// Stop participating.
ldrElect.Stop()
Service Tracker

A recipe to perform client-side service discovery. In a distributed system multiple instances of the same service will be running in order to be highly available. Hence the consumers will need a mechanism to know where those instances are running.

The ServiceTracker can be used to achieve that. The multiple instances of a service will register themselves with the etcd service at an agreed upon path. The clients of that service will instantiate a ServiceTracker recipe in order to discover/track the service instances.

The following sample snippet describes the usage:

// Construct the path to the location where the services would have registered.
servicePath := conn.ConstructPath("services", "service_name")

// Instantiate a service tracker recipe instance.
svcTracker := conn.NewServiceTracker(servicePath)

// Begin tracking.
svcDataCh, _ := svcTracker.Start()

// Listen to the changes.
go func() {
	for svcData := range svcDataCh {
		// svcData.Services field contains a map of all the instances.
	}
}()
...
...
// End tracker.
svcTracker.Stop()
Ephemeral Key

A recipe to create a key in etcd that would be present till the time the service that created the key is alive. This is similar to the ephemeral znodes concept in the Apache Zookeeper world. This is very handy while performing liveness tracking. A service announces its existence by creating a key in etcd with a TTL value set. As long as the service is alive it keeps renewing the TTL of the key. When the service dies the TTL of the key would eventually expire causing the key to be deleted thus notifying the component interested in its liveness.

The following sample snippet describes the usage:

// Construct the path and instantiate an ephemeral key instance.
path := conn.ConstructPath("services/service_name", "instance1")
ephKey := conn.NewEphemeralKey(path)

// Create the key by setting a value and TTL to it.
ttl := 10 * time.Second
errCh, _ := ephKey.Create("value describing the service instance (<IP:Port#>)", ttl)

// Listen for errors.
go func() {
	for err := range errCh {
		// Take appropriate action.
	}
}()
...
...
// Update the key contents.
err := ephKey.Update("some other value")
...
...
// Delete the ephemeral key.
ephKey.Delete()
Observer

A simple recipe that wraps etcd's Watcher construct. All the changes that occur to a given key will be sent back to the caller via a channel. The user can specify the etcd index whence the observation should start.

The following sample snipper describes the usage:

// Instantiate an Observer instance.
obsvr := conn.NewObserver(path)

// Start the observation with recursive flag set to true and waitIndex of 0 (from beginning).
recursive := true
waitIndex := 0
oRespCh, _ := obsvr.Start(waitIndex, recursive)

// Look out for changes.
go func() {
	for oResp := range oRespCh {
		// Take action.
	}
}()
...
...
// Stop the observer.
obsvr.Stop()
Health Monitor

A recipe that provides an API to monitor the health of the etcd cluster. This is done by querying the etcd's "/health" endpoint. The implementation is same as the "cluster-health" command of etcdctl tool. The source can be found here: github.com/coreos/etcd/etcdctl/command/cluster_health.go

The following sample snippet describes the usage:

// Instantiate a health monitor instance.
hMon := conn.NewHealthMonitor()

// Specify whether the monitor needs to run once and if not what the scan frequency should be.
once := false
scanFreq := 10 * time.Second

// Start monitoring.
monCh := hMon.Start(once, scanFreq)

// Look for updates.
go func() {
	for status := range monCh {
		if status == etcdc.ClusterStatusHealthy {
			// Cluster health is good. Carry out regular operations. 
		} else if status == etcdc.ClusterStatusUnhealthy {
			// Cluster health is bad.
		}
	}
}()
...
...
// Stop monitoring the cluster health.
hMon.Stop()

Documentation

Index

Constants

View Source 
const (
	// Timeout value set on every http request sent to the etcd service.
	TIMEOUT_PER_REQUEST = (2 * time.Second)

	// Default TTL value that will be used for keys (used mainly by
	// EphemeralKey and LeaderElector operations).
	TTL_VAL = (10 * time.Second)

	// Default ticker value that will be used to refresh the TTL.
	TTL_REFRESH_TIMEOUT = (TTL_VAL - (1 * time.Second))
)
View Source 
const LE_SLEEP_INTERVAL = 5 * time.Second

Variables

This section is empty.

Functions

func PrepareEndpointList 

func PrepareEndpointList(srvs []string, port int) []string

Description: 

A helper routine to construct an array of strings from the given server
list and port number into the following format 'http://<server>:<port>'.

Parameters: 

@srvs - An array of strings representing the hostname or IP addresses of
        all the servers that make up the etcd service.
@port - Port at which the servers will accept client connections.

Return value:

  1. An array of strings in 'http://<server>:<port>' format.

Types

type ClusterStatus 

type ClusterStatus int32

A type to indicate the status of the cluster. 

const (
	ClusterStatusHealthy ClusterStatus = iota
	ClusterStatusUnhealthy
	ClusterStatusUnknown
)

type ElectionResponse 

type ElectionResponse struct {
	Status LeaderElectorStatus
	Err    error
}

A structure that will be sent back to the caller to notify if the caller has acquired or lost leadership. On success, @err will be nil.

type EphemeralKey 

type EphemeralKey struct {
	// contains filtered or unexported fields
}

A descriptor structure for the Ephemeral key.

func (*EphemeralKey) Create 

func (ek *EphemeralKey) Create(val string, interval time.Duration) (<-chan error, error)

Description: 

A routine that instantiates an ephemeral key by creating one at
@keyPath specified. The routine also sets @val, passed by the user,
as the value to the key and uses @interval as the time to renew
the TTL of the key at regular intervals.

Parameters: 

@val      - User defined value that will be set for the ephemeral key.
@interval - Interval at which the TTL will be renewed.

Return value:

  1. A channel on which errors that occur during TTL renewal will be notified.
  2. Error object describing the error, if any, that occurs during initial setup.

func (*EphemeralKey) Delete 

func (ek *EphemeralKey) Delete()

Description: 

A routine that deletes the EphemeralKey instance. Once stopped an attempt
will be made to manually delete the ephemeral key from etcd namespace.
The deletion is attempted by the go-routine created in Create API.

Parameters: 

None

Return value: 

None

func (*EphemeralKey) Update 

func (ek *EphemeralKey) Update(newVal string) error

Description: 

A routine that updates the value stored in the ephemeral key. It's done by
sending the new value on the @updateCh channel which will be read by the
go-routine created by the Create API. As a side effect the TTL also gets
renewed.

Parameters: 

@newVal - A new user defined value that will be set for the ephemeral key.

Return value:

  1. Error object describing the error.

type EtcdConnector 

type EtcdConnector struct {
	// etcd client API Interface. This is used as an anonymous field.
	client.KeysAPI
	// contains filtered or unexported fields
}

Etcd is a distributed key-value store which can be used for the following purposes: 

[1] To store cluster wide configuration information. Hence the
    name "EtcdConnector".
[2] To achieve distributed coordination.

This structure acts as a handle to the etcd service. The handle can be used to perform all the client operations (like set, get, watch etc...) on the etcd service. It can also be used to make use of some common recipes like leader election, service discovery etc... on the etcd service.

func NewEtcdConnector 

func NewEtcdConnector(servers []string, name string) (*EtcdConnector, error)

Description: 

A constructor routine to instantiate a connection to the etcd service.

Parameters: 

@servers - Array of server host:port pairs to which a client connection
           will be attempted.
@name    - A string that identifies the namespace under which all the
           keys will be created.
           For Ex: If @name is set to 'abc' and a key named 'hosts' is
           created then 'hosts' would be present at '/abc/hosts'.

Return values:

  1. A pointer to an EtcdConnector structure on success, nil otherwise.
  2. An error object on failure, nil otherise.

func (*EtcdConnector) ComputeServerRTT 

func (ec *EtcdConnector) ComputeServerRTT() error

Description: 

A helper routine to approximate the time required for a request to reach
one of the etcd servers. This needs to be accounted as it will be crucial
for picking up a time interval to update the TTL of a key.

At the end of this routine the approximate RTT value will be stored in the
EtcdConnector structure.

Parameters: 

None

Return value:

  1. Corresponding error object, if any.

func (*EtcdConnector) ConstructPath 

func (ec *EtcdConnector) ConstructPath(dir string, key string) string

Description: 

A helper routine to construct a path under the namespace identified by
@name field in EtcdConnector structure.

Parameters: 

@dir - A directory (already existing) under which @key will be created.
@key - A key to be created.

Return value:

  1. A string that takes following form: /@EtcdConnector.name/@dir/@key

func (*EtcdConnector) NewEphemeralKey 

func (ec *EtcdConnector) NewEphemeralKey(path string) *EphemeralKey

Description: 

A constructor routine to instantiate an EphemeralKey structure.

Parameters: 

@path - Path in etcd namespace where the ephemeral key will be created.

Return value:

  1. A pointer to EphemeralKey structure.

func (*EtcdConnector) NewHealthMonitor 

func (ec *EtcdConnector) NewHealthMonitor() *HealthMonitor

Description: 

A constructor routine to instantiate the HealthMonitor structure.

Parameters: 

None

Return value:

  1. A pointer to HealthMonitor structure.

func (*EtcdConnector) NewLeaderElector 

func (ec *EtcdConnector) NewLeaderElector(key, val string, interval time.Duration) LeaderElector

Description: 

A constructor method used to instantiate a leader election participant.

Parameters: 

@key - key path that will be used as a lock for leader election.
@val - Value that will be stored in the key.

NOTE: @val is the value that will be returned back when a follower 

calls the "GetLeader" method. So the user will have to choose
the value in a way that will be easy for the followers to
identify the leader instance. (Example value : <IP addr:Port>)

Return value:

  1. A LeaderElector interface.

func (*EtcdConnector) NewObserver 

func (ec *EtcdConnector) NewObserver(key string) *Observer

Description: 

A constructor method used to instantiate a new Observer structure.

Parameters: 

@key - Path to the key that needs to be observed. This could be a
       directory or a key.

Return value:

  1. A pointer to an Observer structure.

func (*EtcdConnector) NewServiceTracker 

func (ec *EtcdConnector) NewServiceTracker(path string) ServiceTracker

Description: 

A constructor routine to instantiate a service tracker.

Parameters: 

@path - A path in the etcd namespace under which the instances will
        be tracked.

Return value:

  1. A pointer to the ServiceTracker instance.

type HealthMonitor 

type HealthMonitor struct {
	// contains filtered or unexported fields
}

A descriptor structure for the Health Monitor.

func (*HealthMonitor) Start 

func (hm *HealthMonitor) Start(once bool, scanFreq time.Duration) <-chan ClusterStatus

Description: 

A routine that starts the etcd cluster health monitor. The "/health" endpoint of
a client URL shall be probed to check the status of the cluster. The logic of this
routine is very similar to the implementation of "cluster-health" command of the
etcdctl tool provided by CoreOS. It's just that the functionality is being provided
in the form of an API. The caller gets a channel, as a return value, on which the
status will be posted.

Parameters: 

@once     - A flag to indicate to check the status once and exit.
@scanFreq - A time interval after which the status needs to be checked in a loop.
            This field makes sense when @once is set to false.

Return value:

  1. A channel on which ClusterStatus will be notified.

func (*HealthMonitor) Stop 

func (hm *HealthMonitor) Stop()

Description: 

A routine that stops the health monitor. The stop channel will be written
to signal the go-routine to exit.

Parameters: 

None

Return value: 

None

type LeaderElector 

type LeaderElector interface {
	// Start participating in the election process. A channel shall be returned
	// on which election status shall be reported back to the user.
	Start() (<-chan ElectionResponse, error)

	// Stop particpating in the election process.
	Stop()

	// Get the identity of the leader instance. This is a method via which
	// the followers can learn who the current leader is.
	GetLeader() string
}

LeaderElector interface to be used by the users of the recipe.

type LeaderElectorStatus 

type LeaderElectorStatus int32

A type to describe the status of the leader election operation. 

const (
	// Indicates that the caller has become the leader.
	LEStatusLeader LeaderElectorStatus = iota

	// Indicates that the caller has lost leaderhip.
	LEStatusFollower
)

type Observer 

type Observer struct {
	// contains filtered or unexported fields
}

A descriptor structure for the Observer operation.

func (*Observer) Start 

func (o *Observer) Start(waitIndex uint64, recursive bool) (<-chan ObserverResponse, error)

Description: 

A routine that starts the watch on the key identified by @key member
Observer structure. The caller gets a channel, as a return value, on
which ObserverResponse will be sent out.

Parameters: 

@waitIndex - The index of the key from which we intend to wait for changes.
             If waitIndex is zero that means that we wait for any change.
@recursive - A flag to indicate to look for changes recursively in a
             directory structure.

Return values:

  1. A channel on which responses will be sent.
  2. Error info, if any while starting the operation, will be returned.

func (*Observer) Stop 

func (o *Observer) Stop()

Description: 

A routine that stops an Observer. The context with cancellation capability
will be used to stop the observation.

Parameters: 

None

Return value: 

None

type ObserverResponse 

type ObserverResponse struct {
	// An etcd response object received when a watch triggers.
	Response *client.Response

	// An error object, if any.
	Err error
}

A wrapper structure that will be sent back to caller as a response whenever a watch triggers or an error occurs.

type ServiceData 

type ServiceData struct {
	// A map of active services with service name as key and its content
	// as value.
	Services map[string]string

	// Error information, if any.
	Err error
}

A structure that will be sent back to the caller whenever a change is observed under @servicePath.

type ServiceTracker 

type ServiceTracker interface {
	// Start tracking the instances that form a given service. A channel shall
	// be returned on which the details about the service instances will be sent.
	Start() (<-chan ServiceData, error)

	// Stop tracking for changes.
	Stop()
}

ServiceTracker interface to be used by the users of this recipe.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.