awsservice

README

AWS CloudWatch Metrics

Supported AWS services

• ApplicationELB
• CloudFront
• DynamoDB
• DX
• EBS
• EC2
• EC2AutoScaling
• EC2Spot
• ECS
• EFS
• ElastiCache
• ElasticBeanstalk
• ElasticInterface
• ElasticMapReduce
• ElasticTranscoder
• ELB
• ES
• KMS
• Lambda
• NATGateway
• NetworkELB
• RDS
• Route53
• Route53Resolver
• S3
• SNS
• SQS
• TransitGateway

Installation

1. Create a directory for the install. Suggested: mkdir -p /opt/circonus/cloud-agent
2. Download the latest release
3. Unpack the release in the directory created in first step
4. In this directory, create a config folder. Suggested: mkdir /opt/circonus/cloud-agent/etc/aws.d
5. Auto-create a service specific configuration template in the desired format (yaml, toml, or json). Suggested: sbin/circonus-cloud-agentd --enable-aws --aws-example-conf=yaml > /opt/circonus/cloud-agent/etc/aws.d/aws-config.yaml
   • Note, the id in the template is defaulted to the filename. This should be changed to a name that will be unique across all cloud-agents used in Circonus
   • Add the Circonus api key
   • Add the AWS credentials
   • Update settings for the desired AWS services to be monitored
6. Setup as a system service or run in foreground ensuring that --enable-aws is specified

Options

$  sbin/circonus-cloud-agentd -h
The Circonus Cloud Agent collects metrics from cloud infrastructures and fowards them to Circonus.

Usage:
  circonus-cloud-agent [flags]

Flags:
      --aws-conf-dir string         AWS configuration directory (default "/opt/circonus/cloud-agent/etc/aws.d")
      --aws-example-conf string     Show AWS config (json|toml|yaml) and exit
  -c, --config string               config file (default: circonus-cloud-agent.yaml|.json|.toml)
  -d, --debug                       [ENV: CCA_DEBUG] Enable debug messages
      --enable-aws                  Enable AWS metric collection client
  -h, --help                        help for circonus-cloud-agent
      --log-level string            [ENV: CCA_LOG_LEVEL] Log level [(panic|fatal|error|warn|info|debug|disabled)] (default "info")
      --log-pretty                  [ENV: CCA_LOG_PRETTY] Output formatted/colored log lines [ignored on windows]
  -V, --version                     Show version and exit

Configuration

AWS

NOTE: the agent can run in a shared mode with multiple sets of credentials or a local mode where the credentials are kept in an external location and roles are used to identify which credentials to use.

1. Create an IAM group e.g. circonus-cloud-agent (if one does not already exist) with the following permissions:
   1. Required: CloudWatchReadOnlyAccess - to retrieve metrics
   2. Optional: AmazonEC2ReadOnlyAccess - to get list of instances, if EC2 metrics are desired
   3. Optional: AmazonElastiCacheReadOnlyAccess - to get list of cache clusters, if ElastiCache metrics are desired
2. Create an IAM user e.g. circonus-cloud-agent (ensure it is a member of the aforementioned group), make a note of the Access Key Credentials (access key id and secret access key or role and save credentials file locally where circonus-cloud-agent will run, ensure it is accessible by whatever user the circonus-cloud-agent is configured to run as.)

Circonus

1. Use Circonus UI to create or identify an API Token to use
2. Add the key to the config file under the circonus section

AWS settings

For per-configuration file credentials (shared):

• access_key_id
• secret_access_key

or, for credentials in a local file:

• role
• credentials_file

Example configuration

Minimum configuration (for EC2 service):

Credentials in configuration file:

---

id: ...
aws:
  access_key_id: ...
  secret_access_key: ...
circonus:
  key: ...
period: basic
regions:
    - name: us-east-1
      services:
          - namespace: aws/EC2
            disabled: false

Shared credentials using roles:

---

id: ...
aws:
  role: ...
  credentials_file: ...
circonus:
  key: ...
period: basic
regions:
    - name: us-east-1
      services:
          - namespace: aws/EC2
            disabled: false

Documentation

Index

• Constants
• Variables
• type AWS
• type AWSRegion
• type AWSService
  • func New(ctx context.Context) (*AWSService, error)
  • func (svc *AWSService) Enabled() bool
  • func (svc *AWSService) Scan() error
  • func (svc *AWSService) Start() error
• type Config
• type Instance
  • func (inst *Instance) Start() error

Constants

const (
	// KeyEnabled toggles whether the AWS module is active or not.
	KeyEnabled = "aws.enabled"
	// DefaultEnabled defines the default setting.
	DefaultEnabled = false

	// KeyConfDir defines the aws configuration directory.
	KeyConfDir = "aws.conf_dir"

	// KeyConfExample shows an example configuration.
	KeyConfExample = "aws.config_example"
)

Variables

var (
	// DefaultConfDir is the default location of aws configuration files.
	DefaultConfDir = path.Join(defaults.EtcPath, "aws.d")
)

Types

type AWS

type AWS struct {
	//
	// Runs in ONE of two ways: shared or local
	//   shared - multiple different sets of credentials - use AccessKeyID and SecretAccessKey from an IAM role
	//   local - use a role in a local credentials file
	//
	// Mode is swithed based on which is configured, configure only ONE pair of authentication settings
	//
	// see: https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#Value
	// to be used with: https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#NewStaticCredentials
	AccessKeyID     string `json:"access_key_id" toml:"access_key_id" yaml:"access_key_id"`
	SecretAccessKey string `json:"secret_access_key" toml:"secret_access_key" yaml:"secret_access_key"`
	// SessionToken    string `json:"session_token" toml:"session_token" yaml:"session_token"`
	// ProviderName    string `json:"provider_name" toml:"provider_name" yaml:"provider_name"`
	//
	// Local mode - non-shared version, running a local instance of circonus-cloud-agent
	// use with https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#NewSharedCredentials
	Role            string `json:"role" toml:"role" yaml:"role"`
	CredentialsFile string `json:"credentials_file" toml:"credentials_file" yaml:"credentials_file"`
}

AWS defines the credentials to use for AWS.

type AWSRegion

type AWSRegion struct {
	Name     string                    `json:"name" toml:"name" yaml:"name"`             // e.g. us-east-1
	Services []collectors.AWSCollector `json:"services" toml:"services" yaml:"services"` // which services to collectc metrics for in this region
	Tags     circonus.Tags             `json:"tags" toml:"tags" yaml:"tags"`             // region tags (default region:Name)
}

AWSRegion defines a specific aws region from which to collect metrics.

type AWSService

type AWSService struct {
	// contains filtered or unexported fields
}

AWSService defines the AWS cloud service client.

func New

func New(ctx context.Context) (*AWSService, error)

New returns an AWS cloud service metric collector.

func (*AWSService) Enabled

func (svc *AWSService) Enabled() bool

Enabled indicates whether the AWS service is enabled.

func (*AWSService) Scan

func (svc *AWSService) Scan() error

Scan checks the service config directory for configurations and loads them.

func (*AWSService) Start

func (svc *AWSService) Start() error

Start begins collecting metrics from AWS service.

type Config

type Config struct {
	ID       string                 `json:"id" toml:"id" yaml:"id"`                   // unique id for this service client instance, no spaces (ties several things together, short and immutable - logging, check search/create, tags, etc.)
	Regions  []AWSRegion            `json:"regions" toml:"regions" yaml:"regions"`    // list of region specific configurations
	AWS      AWS                    `json:"aws" toml:"aws" yaml:"aws"`                // REQUIRED, aws credentials
	Circonus circonus.ServiceConfig `json:"circonus" toml:"circonus" yaml:"circonus"` // REQUIRED, circonus config: api credentials, check, broker, etc.
	Period   string                 `json:"period" toml:"period" yaml:"period"`       // 'basic' or 'detailed'
	Tags     circonus.Tags          `json:"tags" toml:"tags" yaml:"tags"`             // global tags, added to all metrics
}

Config defines an AWS service instance configuration NOTE: warning - ID must be thought of as immutable - if it changes a new check will be created.

type Instance

type Instance struct {
	sync.Mutex
	// contains filtered or unexported fields
}

Instance AWS SDK/API Instance for fetching cloudwatch metrics and forwarding them to Circonus Note: a Instance has a 1:1 relation with aws:circ - each Instance has (or, may have) a different set of aws and/or circonus credentials.

func (*Instance) Start

func (inst *Instance) Start() error

Start metric collections based on the configured interval - intended to be run in a goroutine (e.g. errgroup).