ec2cluster

README

EC2 Cluster Auto-discovery for GO

This package provides simple cluster auto-discovery for go using EC2 metadata. This should be used to seed a more reliable group membership algorithm, such as github.com/hashicorp/memberlist. Most such algorithms require an initial seed list of nodes, which is what ec2cluster provides.

In this example we use all the EC2 instances tagged with the same value for app as the initial known members for a memberlist cluster:

ec2TagName := "app"
knownMembers, err := DiscoverClusterMembersByTag(&aws.Config{}, ec2TagName)
if err != nil {
    panic(err)
}
cluster := memberlist.Create(memberlist.DefaultLANConfig())
_, err := cluster.Join(knownMembers)
if err != nil {
    panic(err)
}
defer cluster.Leave(time.Second)

When run as a command this program produces output that can be used to set environment variables. You can invoke it like:

eval $(ec2cluster)

The output it produces can be passed to the shell, or parsed:

AVAILABILITY_ZONE="us-west-2a"
REGION="us-west-2"
ADVERTISE_ADDRESS="10.0.10.124"
TAG_AWS_CLOUDFORMATION_STACK_ID="arn:aws:cloudformation:us-west-2:012345678901:stack/myapp/d0a9d5eb-dfed-4455-ab4b-5e7214f337bb"
TAG_APPLICATION="myapp"
TAG_AWS_AUTOSCALING_GROUPNAME="example-Cluster1-Q0YWRWQJC5XL"
TAG_CLUSTER="myapp-cluster1"
TAG_AWS_CLOUDFORMATION_STACK_NAME="myapp"
TAG_AWS_CLOUDFORMATION_LOGICAL_ID="Cluster1"
TAG_NAME="myapp.example.com-master"
ASG_DESIRED_CAPACITY=3
ASG_MIN_SIZE=1
ASG_MAX_SIZE=5
CLUSTER="10.0.10.124 10.0.188.207 10.0.118.6"

Monitoring ASG Lifecycle events

You can also use this tool to monitor autoscaling lifecycle events. To do this, configure your autoscaling group with a lifecycle hook that emits events to an SQS queue. Then invoke ec2cluster --watch which will produce one line of output per event, like:

autoscaling:EC2_INSTANCE_TERMINATING    i-403e6d87

Documentation

Index

• Variables
• func DiscoverAdvertiseAddress() (string, error)
• func DiscoverAvailabilityZone() (string, error)
• func DiscoverClusterMembersByTag(awsSession *session.Session, tagName string) ([]string, error)
• func DiscoverInstanceID() (string, error)
• type Cluster
  • func (s *Cluster) AutoscalingGroup() (*autoscaling.Group, error)
  • func (s *Cluster) Instance() (*ec2.Instance, error)
  • func (s *Cluster) LifecycleEventQueueURL() (string, error)
  • func (s *Cluster) Members() ([]*ec2.Instance, error)
  • func (s *Cluster) WatchLifecycleEvents(queueURL string, cb LifecyleEventCallback) error
• type LifecycleMessage
• type LifecyleEventCallback

Variables

var ErrLifecycleHookNotFound = errors.New("cannot find a suitable lifecycle hook")

Functions

func DiscoverAdvertiseAddress

func DiscoverAdvertiseAddress() (string, error)

DiscoverAdvertiseAddress returns the address that should be advertised for the current node based on the current EC2 instance's private IP address

func DiscoverAvailabilityZone

func DiscoverAvailabilityZone() (string, error)

DiscoverAvailabilityZone returns an AWS availability zone or an empty string if the node is not running in EC2 or cannot reach the EC2 metadata service.

func DiscoverClusterMembersByTag

func DiscoverClusterMembersByTag(awsSession *session.Session, tagName string) ([]string, error)

DiscoverClusterMembersByTag returns a list of the private IP addresses of each EC2 instance having a tag that matches the tag on the currently running instance. The tag is specified by tagName.

For example, if your instances are tagged with a tag named `app` whose value is unique per cluster, then you can invoke:

knownClusterMembers, err := DiscoverClusterMembersByTag(nil, "app")

func DiscoverInstanceID

func DiscoverInstanceID() (string, error)

DiscoverInstanceID returns an AWS instance ID or an empty string if the node is not running in EC2 or cannot reach the EC2 metadata service.

Types

type Cluster

type Cluster struct {
	AwsSession *session.Session
	InstanceID string
	TagName    string
	TagValue   string
	// contains filtered or unexported fields
}

Cluster represents a cluster of AWS nodes. Clusters are a group of EC2 instances that share the same value for one of their EC2 instance tags. You specify the tag as `TagName`. To specify a cluster as all the instances in an autoscaling group, specify `aws:autoscaling:groupName` as the TagName.

If you don't specify InstanceID, then the EC2 metadata service will be used to discover the ID of the currently running instnace.

func (*Cluster) AutoscalingGroup

func (s *Cluster) AutoscalingGroup() (*autoscaling.Group, error)

AutoscalingGroup returns the autoscaling group that the current instance is part of. If the current instance is not a member of any autoscaling group, returns nil and a nil error.

func (*Cluster) Instance

func (s *Cluster) Instance() (*ec2.Instance, error)

Instance returns the currently running EC2 instance.

func (*Cluster) LifecycleEventQueueURL

func (s *Cluster) LifecycleEventQueueURL() (string, error)

LifecycleEventQueueURL inspects the current autoscaling group and returns the URL of the first suitable lifecycle hook queue.

func (*Cluster) Members

func (s *Cluster) Members() ([]*ec2.Instance, error)

Members returns a list of cluster members in order from oldest to youngest.

func (*Cluster) WatchLifecycleEvents

func (s *Cluster) WatchLifecycleEvents(queueURL string, cb LifecyleEventCallback) error

WatchLifecycleEvents monitors a lifecycle event SQS queue and invokes cb for each event. If the callback returns an error, then the lifecycle action is completed with ABANDON. On success, the event is completed with CONTINUE.

type LifecycleMessage

type LifecycleMessage struct {
	AutoScalingGroupName string    `json:",omitempty"`
	Service              string    `json:",omitempty"`
	Time                 time.Time `json:",omitempty"`
	AccountID            string    `json:",omitempty"`
	LifecycleTransition  string    `json:",omitempty"`
	RequestID            string    `json:"RequestId"`
	LifecycleActionToken string    `json:",omitempty"`
	EC2InstanceID        string    `json:"EC2InstanceID"`
	LifecycleHookName    string    `json:",omitempty"`
}

LifecycleMessage represents the message we receive from the autoscaling lifecycle hook when an instance is created or terminated.

type LifecyleEventCallback

type LifecyleEventCallback func(m *LifecycleMessage) (shouldContinue bool, err error)

LifecyleEventCallback is a function that is invoked for each ASG lifecycle event. If the function returns a non-nil error then the message remains in the queue. If `shouldContinue` is true then CompleteLifecycleAction() is invoked with `CONINTUE` otherwise it is invoked with `ABANDON`.