ecs_state

README

Helps manage the state of an ECS cluster to allow for simpler scheduler creation.

This library provides a handy query and state synchronization API so that intelligent placement of ECS Tasks can be quickly made and tracked.

Example usage:

client := ecs.New(&aws.Config{Region: aws.String("us-east-1")})
state := ecs_state.Initialize("default", client, ecs_state.DefaultLogger)
state.RefreshClusterState()
state.RefreshContainerInstanceState()
state.RefreshTaskState()
fmt.Printf("Found Cluster: %+v\n", state.FindClusterByName("default"))
fmt.Printf("Found Locations: %+v\n", state.FindLocationsForTaskDefinition("console-sample-app-static:1"))

When run against the "default" cluster created with a single ContainerInstance by the AWS ECS Getting Started Wizard, you should expect to see the Cluster, ContainerInstance, and Task output, along with an empty array of possible locations to place the first TaskDefinition created. No locations are found because of a port conflict. If you were to scale down the service created in the Getting Started Wizard to 0, running this code again would yield the now available ContainerInstance as a location found.

For more details please see http://williamthurston.com/2015/08/20/create-custom-aws-ecs-schedulers-with-ecs-state.html

Documentation

Overview

The ecs_state package provides a number of methods to track, update, and query the shared state of an AWS ECS cluster. Because ECS exposes the state of the cluster in shared state manner, it is expected for applications monitoring and placing tasks within the ECS cluster to replicate the cluster state into a local working copy and synchronize on occassion.

Author: William Thurston

Index

• Variables
• type Cluster
• type ContainerInstance
• type Logger
  • func (logger *Logger) Debug(args ...interface{})
  • func (logger *Logger) Error(args ...interface{})
  • func (logger *Logger) Info(args ...interface{})
  • func (logger *Logger) Warn(args ...interface{})
• type State
  • func Initialize(clusterName string, ecs_client *ecs.ECS, logger Logger) *State
  • func (state *State) DB() *gorm.DB
  • func (state *State) FindClusterByName(name string) Cluster
  • func (state *State) FindLocationsForTaskDefinition(td string) *[]ContainerInstance
  • func (state *State) FindTaskDefinition(td string) TaskDefinition
  • func (state *State) RefreshClusterState()
  • func (state *State) RefreshContainerInstanceState()
  • func (state *State) RefreshTaskState()
• type Task
• type TaskDefinition

Variables

var DefaultLogger = Logger{log.New(os.Stdout, "\r\n", 0)}

A logger that logs on Stdout provided to quickly get started.

Types

type Cluster

type Cluster struct {
	ARN                string `sql:"size:1024" gorm:"primary_key"`
	Name               string `sql:"unique"`
	Status             string
	ContainerInstances []ContainerInstance
	Tasks              []Task
}

Local representation of an ECS cluster and stored by gorm

type ContainerInstance

type ContainerInstance struct {
	ARN                string `sql:"size:1024" gorm:"primary_key"`
	AgentConnected     bool
	AgentHash          string
	AgentVersion       string
	AgentUpdateStatus  string
	ClusterARN         string `sql:"size:1024;index"`
	DockerVersion      string
	EC2InstanceId      string
	RegisteredCPU      int    `gorm:"column:registered_cpu"`
	RegisteredMemory   int    `gorm:"column:registered_memory"`
	RegisteredTCPPorts string `sql:"size:1024" gorm:"column:registered_tcp_ports"`
	RegisteredUDPPorts string `sql:"size:1024" gorm:"column:registered_udp_ports"`
	RemainingCPU       int    `gorm:"column:remaining_cpu"`
	RemainingMemory    int    `gorm:"column:remaining_memory"`
	RemainingTCPPorts  string `sql:"size:1024" gorm:"column:remaining_tcp_ports"`
	RemainingUDPPorts  string `sql:"size:1024" gorm:"column:remaining_udp_ports"`
	Status             string
	Tasks              []Task

	// Not part of the ECS API
	RefreshTime int
}

Local representation of an ECS ContainerInstance and stored by gorm. Notably, resources and other sub-objects have been placed into their own columns for more robust query capabilities.

type Logger

type Logger struct {
	*log.Logger
}

Allows us to create a default logger if the user does not provide on.

func (*Logger) Debug

func (logger *Logger) Debug(args ...interface{})

Log at a debug level

func (*Logger) Error

func (logger *Logger) Error(args ...interface{})

Log at an error level

func (*Logger) Info

func (logger *Logger) Info(args ...interface{})

Log at an info level

func (*Logger) Warn

func (logger *Logger) Warn(args ...interface{})

Log at a warn level

type State

type State struct {
	// contains filtered or unexported fields
}

The State object provides methods to synchronize and query the state of the ECS cluster.

func Initialize

func Initialize(clusterName string, ecs_client *ecs.ECS, logger Logger) *State

Create a new State object. The clusterName is the cluster to track, ecs_client should be provided by the caller with proper credentials preferably scoped to read only access to ECS APIs, and the logger can use ecs_state.DefaultLogger for output on stdout, or the user can provide a custom logger instead.

func (*State) DB

func (state *State) DB() *gorm.DB

Provides direct access to the database through gorm to allow more advanced queries against state.

func (*State) FindClusterByName

func (state *State) FindClusterByName(name string) Cluster

Load the cluster and all ContainerInstances and Tasks into memory as Go objects.

func (*State) FindLocationsForTaskDefinition

func (state *State) FindLocationsForTaskDefinition(td string) *[]ContainerInstance

Returns all ContainerInstances where the desired TaskDefinition has resources available. Additional filtering or constraints can be added if required.

func (*State) FindTaskDefinition

func (state *State) FindTaskDefinition(td string) TaskDefinition

Resolve and cache locally a Task Definition from either a short string like my_app:1 or a full ARN.

func (*State) RefreshClusterState

func (state *State) RefreshClusterState()

Performs ECS DescribeCluster call on the clusterName provided at Initialization time and updates the local copy of state.

func (*State) RefreshContainerInstanceState

func (state *State) RefreshContainerInstanceState()

Lists and Describes ContainerInstances in the ECS API and stores them in a more queryable form locally. Any ContainerInstances no longer returned by ECS, for example if they have been deregistered, will be removed from the local view of state as well.

func (*State) RefreshTaskState

func (state *State) RefreshTaskState()

Lists and Describes Tasks in the ECS API and stores them in a more queryable form locally. Any Tasks no longer returned by ECS, for example if they have been stopped, will be removed from the local view of state as well.

type Task

type Task struct {
	ARN                  string `sql:"size:1024" gorm:"primary_key"`
	DesiredStatus        string
	LastStatus           string
	StartedBy            string `sql:"index"`
	ClusterARN           string `sql:"size:1024;index"`
	ContainerInstanceARN string `sql:"size:1024;index"`
	TaskDefinitionARN    string `sql:"size:1024;index"`

	// Not part of the ECS API
	RefreshTime int
}

Local representation of an ECS Task and stored by gorm. A number of fields are absent for now as they are not needed to track and update the state of the state of the cluster typically.

type TaskDefinition

type TaskDefinition struct {
	ARN         string `sql:"size:1024" gorm:"primary_key"`
	ShortString string `sql:"unique"`
	Cpu         int
	Memory      int
	TCPPorts    string
	UDPPorts    string
}

Local representation of an ECS TaskDefinition and stored by gorm. Resources are extracted, but the complete definition is ignored.