ecschedule

README

ecschedule

ecschedule is a tool to manage ECS Scheduled Tasks.

Synopsis

% ecschedule [dump|apply|run|diff] -conf ecschedule.yaml -rule $ruleName

Description

The ecschedule manages ECS Schedule tasks using a configuration file (YAML, JSON or Jsonnet format) like following.

region: us-east-1
cluster: clusterName
rules:
- name: taskName1
  description: task 1
  scheduleExpression: cron(30 15 ? * * *)
  taskDefinition: taskDefName
  containerOverrides:
  - name: containerName
    command: [subcommand1, arg]
    environment:
      HOGE: foo
      FUGA: {{ must_env `APP_FUGA` }}
- name: taskName2
  description: task2
  scheduleExpression: cron(30 16 ? * * *)
  taskDefinition: taskDefName2
  containerOverrides:
  - name: containerName2
    command: [subcommand2, arg]

Installation

% go install github.com/Songmu/ecschedule/cmd/ecschedule@latest

GitHub Actions

Action Songmu/ecschedule@main installs ecschedule binary for Linux into /usr/local/bin. This action runs install only.

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: Songmu/ecschedule@main
      - run: |
          ecschedule -conf ecschedule.yaml apply -all

Quick Start

dump configuration YAML

% ecschedule dump --cluster clusterName --region us-east-1 > ecschedule.yaml

edit and adjust configuration file after it.

apply new or updated rule

% ecschedule -conf ecschedule.yaml apply -rule $ruleName

Before you apply it, you can check the diff in the following way.

% ecschedule -conf ecschedule.yaml diff -rule $ruleName

run rule

Execute run subcommand when want execute arbitrary timing.

% ecschedule -conf ecschedule.yaml run -rule $ruleName

Functions

You can use following functions in the configuration file.

• env
  • expand environment variable or using default value
  • {{ env "ENV_NAME" "DEFAULT_VALUE" }}
• must_env
  • expand environment variable
  • {{ must_env "ENV_NAME" }}

inspired by ecspresso.

Plugins

tfstate

tfstate plugin introduces a template function tfstate.

region: us-east-1
cluster: api
role: ecsEventsRole
rules:
- name: hoge-task-name
  description: hoge description
  scheduleExpression: cron(0 0 * * ? *)
  taskDefinition: task1
  group: xxx
  platform_version: 1.4.0
  launch_type: FARGATE
  network_configuration:
    aws_vpc_configuration:
      subnets:
      - {{ tfstate `aws_subnet.private-a.id` }}
      - {{ tfstate `aws_subnet.private-c.id` }}
      security_groups:
      - {{ tfstatef `data.aws_security_group.default['%s'].id` `first` }}
      - {{ tfstatef `data.aws_security_group.default['%s'].id` `second` }}
      assign_public_ip: ENABLED
  containerOverrides:
  - name: container1
    command: ["subcmd", "argument"]
    environment:
      HOGE_ENV: {{ env "DUMMY_HOGE_ENV" "HOGEGE" }}
  dead_letter_config:
    sqs: queue1
  propagateTags: TASK_DEFINITION
plugins:
- name: tfstate
  config:
    path: testdata/terraform.tfstate    # path to tfstate file
      # or url: s3://my-bucket/terraform.tfstate

{{ tfstate "resource_type.resource_name.attr" }} will expand to an attribute value of the resource in tfstate.

{{ tfstatef "resource_type.resource_name['%s'].attr" "index" }} is similar to {{ tfstatef "resource_type.resource_name['index'].attr" }}.
This function is useful to build a resource address with environment variables.

{{ tfstatef `aws_subnet.ecs['%s'].id` (must_env `SERVICE`) }}

Pitfalls

Trap due to rule name uniqueness constraints

ecschedule is designed to guarantee the uniqueness of job definitions by rule name in the configuration file. This causes the following confusing behavior at the moment. Please be careful.

The rule name change causes a problem of garbage definition remaining

If the name of a rule that is already reflected in the configuration file is changed, a new rule with that name is created and the old rule with the old name remains, resulting in an unintended double definition. The only solution is to delete the old rule manually.

Unable to delete rules

If a rule is deleted from the configuration file and then reflected, the rule will not be deleted. The only way to do this is to delete this separately, too manually.

This is because ecschedule only manages a subset of cloudwatch event rules, so it cannot distinguish whether a rule name that does not exist in the configuration file is out of ecschedule management or has been deleted.

Desired solution

There should be designs to implement some state management mechanisms to solve these problems. If you have a good solution, I would be happy to make suggestions.

Author

Songmu

Documentation

Index

• func GetAWSAccountID(sess *session.Session) (string, error)
• func Run(ctx context.Context, argv []string, outStream, errStream io.Writer) error
• type AwsVpcConfiguration
• type BaseConfig
• type Config
  • func LoadConfig(ctx context.Context, r io.Reader, accountID string, confPath string) (*Config, error)
  • func (c *Config) GetRuleByName(name string) *Rule
• type ContainerOverride
• type DeadLetterConfig
• type NetworkConfiguration
• type Plugin
• type Rule
  • func (r *Rule) Apply(ctx context.Context, sess *session.Session, dryRun bool) error
  • func (r *Rule) PutRuleInput() *cloudwatchevents.PutRuleInput
  • func (r *Rule) PutTargetsInput() *cloudwatchevents.PutTargetsInput
  • func (r *Rule) Run(ctx context.Context, sess *session.Session, noWait bool) error
• type Target

Functions

func GetAWSAccountID

func GetAWSAccountID(sess *session.Session) (string, error)

GetAWSAccountID returns id

func Run

func Run(ctx context.Context, argv []string, outStream, errStream io.Writer) error

Run the ecschedule

Types

type AwsVpcConfiguration

type AwsVpcConfiguration struct {
	Subnets        []string `yaml:"subnets" json:"subnets"`
	SecurityGroups []string `yaml:"security_groups,omitempty" json:"security_groups,omitempty"`
	AssinPublicIP  string   `yaml:"assign_public_ip,omitempty" json:"assign_public_ip,omitempty"`
}

AwsVpcConfiguration represents AWS VPC configuration

type BaseConfig

type BaseConfig struct {
	Region    string `yaml:"region" json:"region"`
	Cluster   string `yaml:"cluster" json:"cluster"`
	AccountID string `yaml:"-" json:"-"`
}

BaseConfig baseconfig

type Config

type Config struct {
	Role        string `yaml:"role,omitempty" json:"role,omitempty"`
	*BaseConfig `yaml:",inline" json:",inline"`
	Rules       []*Rule   `yaml:"rules" json:"rules"`
	Plugins     []*Plugin `yaml:"plugins,omitempty" json:"plugins,omitempty"`
	// contains filtered or unexported fields
}

Config config

func LoadConfig

func LoadConfig(ctx context.Context, r io.Reader, accountID string, confPath string) (*Config, error)

LoadConfig loads config

func (*Config) GetRuleByName

func (c *Config) GetRuleByName(name string) *Rule

GetRuleByName gets rule by name

type ContainerOverride

type ContainerOverride struct {
	Name        string            `yaml:"name" json:"name"`
	Command     []string          `yaml:"command,flow" json:"command"` // ,flow
	Environment map[string]string `yaml:"environment,omitempty" json:"environment,omitempty"`
}

ContainerOverride overrids container

type DeadLetterConfig

type DeadLetterConfig struct {
	Sqs string `yaml:"sqs" json:"sqs"`
}

A DeadLetterConfig object that contains information about a dead-letter queue configuration.

type NetworkConfiguration

type NetworkConfiguration struct {
	AwsVpcConfiguration *AwsVpcConfiguration `yaml:"aws_vpc_configuration" json:"aws_vpc_configuration"`
}

NetworkConfiguration represents ECS network configuration

type Plugin

type Plugin struct {
	Name   string                 `yaml:"name"`
	Config map[string]interface{} `yaml:"config"`
}

Plugin the plugin

type Rule

type Rule struct {
	Name               string `yaml:"name" json:"name"`
	Description        string `yaml:"description,omitempty" json:"description,omitempty"`
	ScheduleExpression string `yaml:"scheduleExpression" json:"scheduleExpression"`
	Disabled           bool   `yaml:"disabled,omitempty" json:"disabled,omitempty"` // ENABLE | DISABLE
	*Target            `yaml:",inline" json:",inline"`

	*BaseConfig `yaml:",inline,omitempty"`
}

Rule the rule

func (*Rule) Apply

func (r *Rule) Apply(ctx context.Context, sess *session.Session, dryRun bool) error

Apply the rule

func (*Rule) PutRuleInput

func (r *Rule) PutRuleInput() *cloudwatchevents.PutRuleInput

PutRuleInput puts rule input

func (*Rule) PutTargetsInput

func (r *Rule) PutTargetsInput() *cloudwatchevents.PutTargetsInput

PutTargetsInput puts targets input

func (*Rule) Run

func (r *Rule) Run(ctx context.Context, sess *session.Session, noWait bool) error

Run the rule

type Target

type Target struct {
	TargetID             string                `yaml:"targetId,omitempty" json:"targetId,omitempty"`
	TaskDefinition       string                `yaml:"taskDefinition" json:"taskDefinition"`
	TaskCount            int64                 `yaml:"taskCount,omitempty" json:"taskCount,omitempty"`
	ContainerOverrides   []*ContainerOverride  `yaml:"containerOverrides,omitempty" json:"containerOverrides,omitempty"`
	Role                 string                `yaml:"role,omitempty" json:"role,omitempty"`
	Group                string                `yaml:"group,omitempty" json:"group,omitempty"`
	LaunchType           string                `yaml:"launch_type,omitempty" json:"launch_type,omitempty"`
	PlatformVersion      string                `yaml:"platform_version,omitempty" json:"platform_version,omitempty"`
	NetworkConfiguration *NetworkConfiguration `yaml:"network_configuration,omitempty" json:"network_configuration,omitempty"`
	DeadLetterConfig     *DeadLetterConfig     `yaml:"dead_letter_config,omitempty" json:"dead_letter_config,omitempty"`
	PropagateTags        *string               `yaml:"propagateTags,omitempty" json:"propagateTags,omitempty"`
}

Target cluster