awsx

README

awsx

Package provides Go AWS utilities that I typically have to repeat and use often.

Install

go get -u github.com/routebyintuition/awsx

Usage

For standard AWS connectivity, we have the capability to create a new session based upon the various authentication mechanisms built into the AWS Go SDK.

In the below example, we are using the Amazon EC2 credentials to create a new session:

input := &s3.ListBucketsInput{}

a := awsx.NewAWS().EnablePanic().WithInstanceRole()
a.SetRegion("us-east-1")
a.SetSession()

s3 := s3.New(a.Session)

result, err := s3.ListBuckets(input)
if err != nil {
    fmt.Println("Error: ", err)
    os.Exit(1)
}

fmt.Println(result)

RediGo

If you would like to grab the Redis ElastiCache endpoints for the primary Redis endpoint, the read-only endpoints, or the cluster configuration endpoint for use with a library like go-redis, you can see examples below:

a := awsx.NewAWS().EnablePanic().WithFile()
a.SetRegion("us-west-2")
a.SetSession()

endpoint, err := a.GetRedisPrimaryEndpoint("cluster-name")
if err != nil {
    fmt.Println(err)
}

redisPool := redis.NewPool(func() (redis.Conn, error) {
    c, err := redis.Dial("tcp", endpoint.PrimaryString())

    if err != nil {
        return nil, err
    }

    return c, err
}, 100)

The above example works for standard clusters that have Redis Cluster Mode disabled. You can check for the cluster mode with:

fmt.Println("Primary Endpoint: ", endpoint.PrimaryString())

if endpoint.ClusterEnabled {
    fmt.Println("Cluster Mode Enabled")
} else {
    fmt.Println("Cluster Mode Disabled")
}

We can also pull out the read replicas for their own connections to read:

endpoint, err := a.GetRedisPrimaryEndpoint("redis-cluster")

if err != nil {
    fmt.Println(err)
}

if endpoint.ClusterEnabled {
    fmt.Println("Cluster Mode Enabled")
    fmt.Println("Cluster Configuration Endpoint: ", endpoint.ClusterConfigString())
} else {
    fmt.Println("Cluster Mode Disabled")
    fmt.Println("Primary Endpoint: ", endpoint.PrimaryString())
}

if endpoint.ReadReplicas {
    fmt.Println("Read Replicas available: ")
    for _, v := range endpoint.ReadEndpoints {
        fmt.Println("Read Replica: ", v)
    }
}

/*
Outputs:
Cluster Mode Disabled
Primary Endpoint:  redis-cluster.XXXXXX.ng.0001.XXXX.cache.amazonaws.com:6379
Read Replicas available:
Read Replica:  redis-cluster-001.XXXXXX.0001.XXXX.cache.amazonaws.com:6379
Read Replica:  redis-cluster-002.XXXXXX.0001.XXXX.cache.amazonaws.com:6379
*/

Redis Cluster

You can also use this tool to find your cluster configuration endpoint for use with Redis cluster:

endpoint, err := a.GetRedisPrimaryEndpoint("cluster-name")
if err != nil {
    fmt.Println(err)
}

if endpoint.ClusterEnabled {
    fmt.Println("Cluster Mode Enabled")
    fmt.Println("Cluster Configuration Endpoint: ", endpoint.ClusterConfigString())
} else {
    fmt.Println("Cluster Mode Disabled")
    fmt.Println("Primary Endpoint: ", endpoint.PrimaryString())
}

Additional Information

Original connection methods used from https://github.com/C2FO/vfs with the AWS connection implementation for the S3 io.Writer.

To Do

• Add cloudwatch IO writer
• Add RDS master/replica lookups - MySQL
• Add RDS MySQL IAM auth
• Add RDS master/replica lookups - PostgreSQL
• Add PostgreSQL IAM auth
• Add Redis cluster init
• Add Memcached server lookup

Documentation

Index

• type Config
  • func NewAWS() *Config
  • func (a *Config) DisablePanic() *Config
  • func (a *Config) EnablePanic() *Config
  • func (a *Config) GetECClient() *elasticache.ElastiCache
  • func (a *Config) GetECClusterDetails(cluster string) (*elasticache.DescribeCacheClustersOutput, error)
  • func (a *Config) GetECReplicationGroup(cluster string) (*elasticache.DescribeReplicationGroupsOutput, int)
  • func (a *Config) GetRedisAllEndpoints(cluster string) (*RedisEndpoints, error)
  • func (a *Config) GetRedisClusterEndpoint(cluster string) (*RedisEndpoint, error)
  • func (a *Config) GetRedisPrimaryEndpoint(cluster string) (*RedisEndpoints, error)
  • func (a *Config) GetSession() *session.Session
  • func (a *Config) SetECClient() *Config
  • func (a *Config) SetEndpoint(endpoint string) *Config
  • func (a *Config) SetProfile(profile string) *Config
  • func (a *Config) SetRegion(region string) *Config
  • func (a *Config) SetSession() *Config
  • func (a *Config) WithAllProviders() *Config
  • func (a *Config) WithEnv() *Config
  • func (a *Config) WithFile() *Config
  • func (a *Config) WithInstanceRole() *Config
  • func (a *Config) WithStatic() *Config
• type RedisEndpoint
  • func (re *RedisEndpoint) String() string
• type RedisEndpoints
  • func (res *RedisEndpoints) ClusterConfigString() string
  • func (res *RedisEndpoints) PrimaryString() string
  • func (res *RedisEndpoints) Readers() []string
  • func (res *RedisEndpoints) String() string
• type Services

Types

type Config

type Config struct {
	Region       string // should set AWS region used or a default is used
	Role         string // optional: only if using to assume an AWS role
	AccessKey    string // optional: only used if requiring AWS access key/secret key authentication
	SecretKey    string // optional: only used if requiring AWS access key/secret key authentication
	SessionToken string // optional: only used if requiring AWS access key/secret key authentication
	Endpoint     string // optional: use a specified endpoint for calls
	CredFile     string // optional: credentials file to use
	Profile      string // optional: which credential profile to utilize
	Providers    []credentials.Provider
	Session      *session.Session
	Service      *Services
	ServiceSts   *Services
	// contains filtered or unexported fields
}

Config is the configuration definition for our AWS services.

func NewAWS

func NewAWS() *Config

NewAWS creates a new Config struct and populates it with an empty provider chain

func (*Config) DisablePanic

func (a *Config) DisablePanic() *Config

DisablePanic will set panicOnErr to false so that we do not panic the application in care there is an error

func (*Config) EnablePanic

func (a *Config) EnablePanic() *Config

EnablePanic will set panicOnErr to true so that if we hit any errors, the application will panic out

func (*Config) GetECClient

func (a *Config) GetECClient() *elasticache.ElastiCache

GetECClient returns a client for use with AWS Elasticache

func (*Config) GetECClusterDetails

func (a *Config) GetECClusterDetails(cluster string) (*elasticache.DescribeCacheClustersOutput, error)

GetECClusterDetails provides the initial call to describe the identified cluster

func (*Config) GetECReplicationGroup

func (a *Config) GetECReplicationGroup(cluster string) (*elasticache.DescribeReplicationGroupsOutput, int)

GetECReplicationGroup gathers information about the elasticache replication groups

func (*Config) GetRedisAllEndpoints

func (a *Config) GetRedisAllEndpoints(cluster string) (*RedisEndpoints, error)

GetRedisAllEndpoints returns type RedisEndpoints populated with either a single primary redis endpoint or also including a slice of endpoints for the read replica list

func (*Config) GetRedisClusterEndpoint

func (a *Config) GetRedisClusterEndpoint(cluster string) (*RedisEndpoint, error)

GetRedisClusterEndpoint returns a string representation of the cluster endpoint host ane port for use with Redigo and go-redis as host:port This value is the configuration endpoint from elasticache

func (*Config) GetRedisPrimaryEndpoint

func (a *Config) GetRedisPrimaryEndpoint(cluster string) (*RedisEndpoints, error)

GetRedisPrimaryEndpoint returns a string representation of the cluster endpoint host and port for use with redigo and go-redis This ONLY returns the primary endpoint used for read/write operations

func (*Config) GetSession

func (a *Config) GetSession() *session.Session

GetSession creates a new session based upon the Config struct we built using the above functions

func (*Config) SetECClient

func (a *Config) SetECClient() *Config

SetECClient returns a client for use with AWS Elasticache

func (*Config) SetEndpoint

func (a *Config) SetEndpoint(endpoint string) *Config

SetEndpoint sets the endpoint to use if this is a custom value

func (*Config) SetProfile

func (a *Config) SetProfile(profile string) *Config

SetProfile sets the profile name to be used with authentication

func (*Config) SetRegion

func (a *Config) SetRegion(region string) *Config

SetRegion sets the AWS region to use with the service calls

func (*Config) SetSession

func (a *Config) SetSession() *Config

SetSession calls GetSession and sets the session return as a struct param

func (*Config) WithAllProviders

func (a *Config) WithAllProviders() *Config

WithAllProviders provides a chain of credentials for connectivity

func (*Config) WithEnv

func (a *Config) WithEnv() *Config

WithEnv adds the environment provider to the credential chain so that if AWS environment credentials are available, they will be used for auth

func (*Config) WithFile

func (a *Config) WithFile() *Config

WithFile adds the credentials file to the provider chain so that during authentication, the application will look for the credentials file inthe default locations. In addition to the default locations, if a.awsCredFile is set, we will look at that location for the AWS credentials. If a.awsProfile is set, this will enable the use of a profile name other than default

func (*Config) WithInstanceRole

func (a *Config) WithInstanceRole() *Config

WithInstanceRole adds the credentials from the EC2 instance obtained from the metadata service to the provider list.

func (*Config) WithStatic

func (a *Config) WithStatic() *Config

WithStatic adds a static credential provider to the provider chain

type RedisEndpoint

type RedisEndpoint struct {
	Host  string
	Port  string
	Slots string
}

RedisEndpoint provides the structure of each endpoint entry

func (*RedisEndpoint) String

func (re *RedisEndpoint) String() string

String provides the string representation of the host and port for use in libraries like redigo and go-redis

type RedisEndpoints

type RedisEndpoints struct {
	// The primary endpoint string
	Primary          *RedisEndpoint
	ClusterConfig    *RedisEndpoint
	ReadEndpoints    []*RedisEndpoint
	ReplicationGroup bool
	ReadReplicas     bool
	ClusterEnabled   bool
}

RedisEndpoints provides an identifier for a primary endpoint and a slice of read endpoints

func (*RedisEndpoints) ClusterConfigString

func (res *RedisEndpoints) ClusterConfigString() string

ClusterConfigString provides the cluster configuration endpoint for Redis Cluster if it is in use. Otherwise, an empty string

func (*RedisEndpoints) PrimaryString

func (res *RedisEndpoints) PrimaryString() string

PrimaryString provides the string representation of the host and port for use in libraries like redigo and go-redis of the primary endpoint

func (*RedisEndpoints) Readers

func (res *RedisEndpoints) Readers() []string

Readers returns a string slice of each read associated with the redis cluster These are each endpoints that can be used for read connections

func (*RedisEndpoints) String

func (res *RedisEndpoints) String() string

String provides the string representation of all endpoints in JSON format

type Services

type Services struct {
	Rds *rds.RDS
	Ec  *elasticache.ElastiCache
}

Services stores the used client types so I don't have to remember to do that.