aws

package
v0.7.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 7, 2024 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Package aws implements driver

Index

Constants

View Source 
const HostReserved = "reserved"

HostReserved - custom status to set in the host for simplifying parallel ops in between the updates

Variables

This section is empty.

Functions

This section is empty.

Types

type Config 

type Config struct {
	Region    string `json:"region"`     // AWS Region to connect to
	KeyID     string `json:"key_id"`     // AWS AMI Key ID
	SecretKey string `json:"secret_key"` // AWS AMI Secret Key

	// Optional
	AccountIDs   []string          `json:"account_ids"`   // AWS Trusted account IDs to filter vpc, subnet, sg, images, snapshots...
	InstanceTags map[string]string `json:"instance_tags"` // AWS Instance tags to use when this node provision them

	// Manage the AWS dedicated hosts to keep them busy and deallocate when not needed
	// Key of the map is name of the pool - will be used for identification of the pool
	DedicatedPool map[string]DedicatedPoolRecord `json:"dedicated_pool"`

	// Various options to not hardcode the important numbers
	SnapshotCreateWait util.Duration `json:"snapshot_create_wait"` // Maximum wait time for snapshot availability (create), default: 2h
	ImageCreateWait    util.Duration `json:"image_create_wait"`    // Maximum wait time for image availability (create/copy), default: 2h
}

Config - node driver configuration

func (*Config) Apply 

func (c *Config) Apply(config []byte) error

Apply takes json and applies it to the config structure

func (*Config) Validate 

func (c *Config) Validate() (err error)

Validate makes sure the config have the required defaults & that the required fields are set

type DedicatedPoolRecord added in v0.7.3 

type DedicatedPoolRecord struct {
	Type string `json:"type"` // Instance type handled by the dedicated hosts pool (example: "mac2.metal")
	Zone string `json:"zone"` // Where to allocate the dedicated host (example: "us-west-2c")
	Max  uint   `json:"max"`  // Maximum dedicated hosts to allocate (they sometimes can handle more than 1 capacity slot)

	// Is a special optimization for the Mac dedicated hosts to send them in [scrubbing process] to
	// save money when we can't release the host due to Apple's license of [24 hours] min limit.
	//
	// Details:
	//
	// Apple forces AWS and any of their customers to keep the Mac dedicated hosts allocated for at
	// least [24 hours]. So after allocation you have no way to release the dedicated host even if
	// you don't need it. This makes the mac hosts very pricey for any kind of dynamic allocation.
	// In order to workaround this issue - Aquarium implements optimization to keep the Mac hosts
	// busy with [scrubbing process], which is triggered after the instance stop or termination and
	// puts Mac host in pending state for 1-2hr. That's the downside of optimization, because you
	// not be able to use the machine until it will become available again.
	//
	// That's why this ScrubbingDelay config exists - we need to give Mac host some time to give
	// the workload a chance to utilize the host. If it will not be utilized in this duration - the
	// manager will start the scrubbing process. When the host become old enough - the manager will
	// release it to clean up space for new fresh mac in the roster.
	//
	// * When this option is unset or 0 - no optimization is enabled.
	// * When it's set - then it's a duration to stay idle and then allocate and terminate empty
	// instance to trigger scrubbing.
	//
	// Current implementation is attached to state update, which could be API consuming, so this
	// duration should be >= 1 min, otherwise API requests will be too often.
	//
	// [24 hours]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-mac-instances.html#mac-instance-considerations
	// [scrubbing process]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mac-instance-stop.html
	ScrubbingDelay util.Duration `json:"scrubbing_delay"`
}

DedicatedPoolRecord stores the configuration of AWS dedicated pool of particular type to manage aws ec2 allocate-hosts --availability-zone "us-west-2c" --auto-placement "on" --host-recovery "off" --host-maintenance "off" --quantity 1 --instance-type "mac2.metal"

type Driver 

type Driver struct {
	// contains filtered or unexported fields
}

Driver implements drivers.ResourceDriver interface

func (*Driver) Allocate 

func (d *Driver) Allocate(def types.LabelDefinition, metadata map[string]any) (*types.Resource, error)

Allocate Instance with provided image

It selects the AMI and run instance Uses metadata to fill EC2 instance userdata

func (*Driver) AvailableCapacity added in v0.6.0 

func (d *Driver) AvailableCapacity(_ types.Resources, def types.LabelDefinition) int64

AvailableCapacity allows Fish to ask the driver about it's capacity (free slots) of a specific definition

func (*Driver) Deallocate 

func (d *Driver) Deallocate(res *types.Resource) error

Deallocate the resource

func (*Driver) GetTask added in v0.6.0 

func (d *Driver) GetTask(name, options string) drivers.ResourceDriverTask

GetTask returns task struct by name

func (*Driver) IsRemote added in v0.6.0 

func (*Driver) IsRemote() bool

IsRemote needed to detect the out-of-node resources managed by this driver

func (*Driver) Name 

func (*Driver) Name() string

Name returns name of the driver

func (*Driver) Prepare 

func (d *Driver) Prepare(config []byte) error

Prepare initializes the driver

func (*Driver) Status 

func (d *Driver) Status(res *types.Resource) (string, error)

Status shows status of the resource

func (*Driver) ValidateDefinition 

func (*Driver) ValidateDefinition(def types.LabelDefinition) error

ValidateDefinition checks LabelDefinition is ok

type Factory added in v0.7.3 

type Factory struct{}

Factory implements drivers.ResourceDriverFactory interface

func (*Factory) Name added in v0.7.3 

func (*Factory) Name() string

Name shows name of the driver factory

func (*Factory) NewResourceDriver added in v0.7.3 

func (*Factory) NewResourceDriver() drivers.ResourceDriver

NewResourceDriver creates new resource driver

type Options added in v0.6.0 

type Options struct {
	Image         string            `json:"image"`          // ID/Name of the image you want to use (name that contains * is usually a bad idea for reproducibility)
	InstanceType  string            `json:"instance_type"`  // Type of the instance from aws available list
	SecurityGroup string            `json:"security_group"` // ID/Name of the security group to use for the instance
	Tags          map[string]string `json:"tags"`           // Tags to add during instance creation
	EncryptKey    string            `json:"encrypt_key"`    // Use specific encryption key for the new disks
	Pool          string            `json:"pool"`           // Use machine from dedicated pool, otherwise will try to use one with auto-placement

	UserDataFormat string `json:"userdata_format"` // If not empty - will store the resource metadata to userdata in defined format
	UserDataPrefix string `json:"userdata_prefix"` // Optional if need to add custom prefix to the metadata key during formatting

	// TaskImage options
	TaskImageName       string `json:"task_image_name"`        // Create new image with defined name + "-DATE.TIME" suffix
	TaskImageEncryptKey string `json:"task_image_encrypt_key"` // KMS Key ID or Alias in format "alias/<name>" if need to re-encrypt the newly created AMI snapshots
}

Options for label definition

Example: 

image: ami-abcdef123456
instance_type: c6a.4xlarge
security_group: sg-abcdef123456
tags:
  somekey: somevalue

func (*Options) Apply added in v0.6.0 

func (o *Options) Apply(options util.UnparsedJSON) error

Apply takes json and applies it to the options structure

func (*Options) Validate added in v0.6.0 

func (o *Options) Validate() error

Validate makes sure the options have the required defaults & that the required fields are set

type TaskImage added in v0.7.4 

type TaskImage struct {
	*types.ApplicationTask `json:"-"` // Info about the requested task
	*types.LabelDefinition `json:"-"` // Info about the used label definition
	*types.Resource        `json:"-"` // Info about the processed resource

	Full bool `json:"full"` // Make full (all disks including connected disks), or just the root OS disk image
	// contains filtered or unexported fields
}

TaskImage stores the task data

func (*TaskImage) Clone added in v0.7.4 

func (t *TaskImage) Clone() drivers.ResourceDriverTask

Clone makes a copy of the initial task to execute

func (*TaskImage) Execute added in v0.7.4 

func (t *TaskImage) Execute() (result []byte, err error)

Execute - Image task could be executed during ALLOCATED & DEALLOCATE ApplicationStatus

func (*TaskImage) Name added in v0.7.4 

func (*TaskImage) Name() string

Name returns name of the task

func (*TaskImage) SetInfo added in v0.7.4 

func (t *TaskImage) SetInfo(task *types.ApplicationTask, def *types.LabelDefinition, res *types.Resource)

SetInfo defines information of the environment

type TaskSnapshot added in v0.6.0 

type TaskSnapshot struct {
	*types.ApplicationTask `json:"-"` // Info about the requested task
	*types.LabelDefinition `json:"-"` // Info about the used label definition
	*types.Resource        `json:"-"` // Info about the processed resource

	Full bool `json:"full"` // Make full (all disks including OS image), or just the additional disks snapshot
	// contains filtered or unexported fields
}

TaskSnapshot stores the task data

func (*TaskSnapshot) Clone added in v0.6.0 

func (t *TaskSnapshot) Clone() drivers.ResourceDriverTask

Clone makes a copy of the initial task to execute

func (*TaskSnapshot) Execute added in v0.6.0 

func (t *TaskSnapshot) Execute() (result []byte, err error)

Execute - Snapshot task could be executed during ALLOCATED & DEALLOCATE ApplicationStatus

func (*TaskSnapshot) Name added in v0.6.0 

func (*TaskSnapshot) Name() string

Name returns name of the task

func (*TaskSnapshot) SetInfo added in v0.6.0 

func (t *TaskSnapshot) SetInfo(task *types.ApplicationTask, def *types.LabelDefinition, res *types.Resource)

SetInfo defines information of the environment

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.