s3

Documentation

Index

• func Key(prefix, firstSeq, lastSeq string) string
• type Emitter
  • func NewEmitter(bucket, region string) *Emitter
  • func (e *Emitter) Emit(s3Key string, b io.ReadSeeker) error
• type ManifestEmitter
  • func (e ManifestEmitter) Emit(s3Key string, b io.ReadSeeker) error

Functions

func Key

func Key(prefix, firstSeq, lastSeq string) string

Types

type Emitter

type Emitter struct {
	Bucket string
	Region string
	// contains filtered or unexported fields
}

Emitter stores data in S3 bucket.

The use of this struct requires the configuration of an S3 bucket/endpoint. When the buffer is full, this struct's Emit method adds the contents of the buffer to S3 as one file. The filename is generated from the first and last sequence numbers of the records contained in that file separated by a dash. This struct requires the configuration of an S3 bucket and endpoint.

func NewEmitter

func NewEmitter(bucket, region string) *Emitter

func (*Emitter) Emit

func (e *Emitter) Emit(s3Key string, b io.ReadSeeker) error

Emit is invoked when the buffer is full. This method emits the set of filtered records.

type ManifestEmitter

type ManifestEmitter struct {
	OutputStream string
	Bucket       string
	Prefix       string
}

An implementation of Emitter that puts event data on S3 file, and then puts the S3 file path onto the output stream for processing by manifest application.

func (ManifestEmitter) Emit

func (e ManifestEmitter) Emit(s3Key string, b io.ReadSeeker) error