state

Documentation

Index

• type DownloadedObject
• type DynamoDBStater
  • func NewDynamoDBStater(session *session.Session, tableName string) (*DynamoDBStater, error)
  • func (d *DynamoDBStater) IsProcessed(object string) (bool, error)
  • func (d *DynamoDBStater) SetProcessed(s3object string) error
• type FileStater
  • func NewFileStater(stateDir, service string) *FileStater
  • func (f *FileStater) IsProcessed(object string) (bool, error)
  • func (f *FileStater) ProcessedObjects() (map[string]time.Time, error)
  • func (f *FileStater) SetProcessed(object string) error
• type Record
• type Stater

Types

type DownloadedObject

type DownloadedObject struct {
	Object, Filename string
}

Used to communicate between the various pieces which are relying on state information.

type DynamoDBStater

type DynamoDBStater struct {
	TableName string
	// contains filtered or unexported fields
}

func NewDynamoDBStater

func NewDynamoDBStater(session *session.Session, tableName string) (*DynamoDBStater, error)

func (*DynamoDBStater) IsProcessed

func (d *DynamoDBStater) IsProcessed(object string) (bool, error)

func (*DynamoDBStater) SetProcessed

func (d *DynamoDBStater) SetProcessed(s3object string) error

type FileStater

type FileStater struct {
	*sync.Mutex
	StateDir         string
	Service          string
	BackfillInterval time.Duration
}

FileStater is an implementation for indicating processing state using the local filesystem for backing storage.

func NewFileStater

func NewFileStater(stateDir, service string) *FileStater

func (*FileStater) IsProcessed

func (f *FileStater) IsProcessed(object string) (bool, error)

func (*FileStater) ProcessedObjects

func (f *FileStater) ProcessedObjects() (map[string]time.Time, error)

func (*FileStater) SetProcessed

func (f *FileStater) SetProcessed(object string) error

type Record

type Record struct {
	S3Object string
	Time     time.Time
}

Used for unmarshaling and adding objects to DynamoDB

type Stater

type Stater interface {
	// SetProcessed indicates that downloading, processing, and sending the
	// object to Honeycomb has been completed successfully.
	SetProcessed(object string) error

	IsProcessed(object string) (bool, error)
}

Stater lets us gain insight into the current state of object processing. It could be backed by the local filesystem, cloud abstractions such as DynamoDB, consistent value stores like etcd, etc.