spotfeed

Documentation

Overview

Package spotfeed is used for querying spot-data-feeds provided by AWS. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-data-feeds.html for a description of the spot data feed format.

This package provides two interfaces for interacting with the AWS spot data feed format for files hosted on S3.

1. Fetch - makes a single blocking call to fetch feed files for some historical period, then parses and returns the results as a single slice.
2. Stream - creates a goroutine that asynchronously checks (once per 30mins by default) the specified S3 location for new spot data feed files (and sends parsed entries into a channel provided to the user at invocation).

This package also provides a LocalLoader which can perform a Fetch operation against feed files already downloaded to local disk. This is often useful for analyzing spot usage over long periods of time, since the download phase can take some time.

Index

• Variables
• type Cost
• type Entry
  • func ParseFeedFile(feed io.Reader, accountId string) ([]*Entry, error)
• type Loader
  • func NewLocalLoader(path string, log *log.Logger, accountId string, startTime, endTime *time.Time, ...) Loader
  • func NewS3Loader(bucket, rootURI string, client s3iface.S3API, log *log.Logger, ...) Loader
  • func NewSpotFeedLoader(sess *session.Session, log *log.Logger, startTime, endTime *time.Time, ...) (Loader, error)
• type Period
• type Querier
  • func NewQuerier(ctx context.Context, l Loader) (Querier, error)

Variables

var ErrMissingData = fmt.Errorf("missing data")

ErrMissingData is the error returned if there is no data for the query time period.

Types

type Cost

type Cost struct {
	// Period defines the time period for which this cost is applicable.
	Period
	// ChargeUSD is the total charge over the time period specified by Period.
	ChargeUSD float64
}

type Entry

type Entry struct {

	// AccountId is a 12-digit account number (ID) that specifies the AWS account
	// billed for this spot instance-hour.
	AccountId string

	// Timestamp is used to determine the price charged for this instance usage.
	// It is not at the hour boundary but within the hour specified by the title of
	// the data feed file that contains this Entry.
	Timestamp time.Time

	// UsageType is the type of usage and instance type being charged for. For
	// m1.small Spot Instances, this field is set to SpotUsage. For all other
	// instance types, this field is set to SpotUsage:{instance-type}. For
	// example, SpotUsage:c1.medium.
	UsageType string

	// Instance is the instance type being charged for and is a member of the
	// set of information provided by UsageType.
	Instance string

	// Operation is the product being charged for. For Linux Spot Instances,
	// this field is set to RunInstances. For Windows Spot Instances, this
	// field is set to RunInstances:0002. Spot usage is grouped according
	// to Availability Zone.
	Operation string

	// InstanceID is the ID of the Spot Instance that generated this instance
	// usage.
	InstanceID string

	// MyBidID is the ID for the Spot Instance request that generated this instance usage.
	MyBidID string

	// MyMaxPriceUSD is the maximum price specified for this Spot Instance request.
	MyMaxPriceUSD float64

	// MarketPriceUSD is the Spot price at the time specified in the Timestamp field.
	MarketPriceUSD float64

	// ChargeUSD is the price charged for this instance usage.
	ChargeUSD float64

	// Version is the version included in the data feed file name for this record.
	Version int64
	// contains filtered or unexported fields
}

Entry corresponds to a single line in a Spot Instance data feed file. The Spot Instance data feed files are tab-delimited. Each line in the data file corresponds to one instance hour and contains the fields listed in the following table. The AccountId field is not specified for each individual entry but is given as a prefix in the name of the spot data feed file.

func ParseFeedFile

func ParseFeedFile(feed io.Reader, accountId string) ([]*Entry, error)

type Loader

type Loader interface {
	// Fetch performs a single blocking call to fetch a discrete set of Entry objects.
	Fetch(ctx context.Context, tolerateErr bool) ([]*Entry, error)

	// Stream asynchronously retrieves, parses and sends Entry objects on the returned channel.
	// To graciously terminate the goroutine managing the Stream, the client terminates the given context.
	Stream(ctx context.Context, tolerateErr bool) (<-chan *Entry, error)
}

Loader provides an API for pulling Spot Data Feed Entry objects from some repository. The tolerateErr parameter configures how the Loader responds to errors parsing individual files or entries; if true, the Loader will continue to parse and yield Entry objects if an error is encountered during parsing.

func NewLocalLoader

func NewLocalLoader(path string, log *log.Logger, accountId string, startTime, endTime *time.Time, version int64) Loader

NewLocalLoader returns a Loader which fetches feed files from a path on the local filesystem. It does not support the Stream API.

func NewS3Loader

func NewS3Loader(bucket, rootURI string, client s3iface.S3API, log *log.Logger, accountId string, startTime, endTime *time.Time, version int64) Loader

NewS3Loader returns a Loader which queries the S3 API for feed files. It supports the Fetch and Stream APIs.

func NewSpotFeedLoader

func NewSpotFeedLoader(sess *session.Session, log *log.Logger, startTime, endTime *time.Time, version int64) (Loader, error)

NewSpotFeedLoader returns a Loader which queries the spot data feed subscription using the given session and returns a Loader which queries the S3 API for feed files (if a subscription does exist). NewSpotFeedLoader will return an error if the spot data feed subscription is missing.

type Period

type Period struct {
	Start, End time.Time
}

Period is a time period with a start and end time.

type Querier

type Querier interface {

	// Query computes the cost charged for the given instanceId for given time period
	// assuming that terminated was the time at which the instance was terminated.
	//
	// It is not required to specify terminated time.  Specifying it only impacts cost
	// calculations for a time period that overlaps the last partial hour of the instance's lifetime.
	//
	// For example, if the instance was running only for say 30m in the last partial hour, and if the
	// desired time period overlaps say the first 15m of that hour, then one must specify
	// terminated time to compute the cost correctly.  In this example, not specifying terminated time
	// would result in a cost higher than actual (ie for the entire last 30 mins instead of only 15 mins).
	//
	// If the given time period spans beyond the instance's actual lifetime, the returned cost will
	// yet only reflect the lifetime cost.  While the returned cost will have the correct start time,
	// the correct end time will be set only if terminated time is provided.
	//
	// Query will return ErrMissingData if it has no data for the given instanceId or
	// if it doesn't have data overlapping the given time period.
	Query(instanceId string, p Period, terminated time.Time) (Cost, error)
}

Querier provides the ability to query for costs.

func NewQuerier

func NewQuerier(ctx context.Context, l Loader) (Querier, error)

NewQuerier fetches data from the given loader and returns a Querier based on the returned data.