destinations

Documentation

Index

• Constants
• type Destination
  • func CreateS3Destination(jsonAPI jsoniter.API) Destination
• type S3Destination
  • func (d *S3Destination) SendEvents(parsedEventChannel chan *parsers.Result, errChan chan error)

Constants

const (

	// The timestamp layout used in the S3 object key filename part with second precision: yyyyMMddTHHmmssZ
	S3ObjectTimestampLayout = "20060102T150405Z"
)

Types

type Destination

type Destination interface {
	SendEvents(parsedEventChannel chan *parsers.Result, errChan chan error)
}

Destination defines the interface that all Destinations should follow

func CreateS3Destination

func CreateS3Destination(jsonAPI jsoniter.API) Destination

type S3Destination

type S3Destination struct {
	// contains filtered or unexported fields
}

S3Destination sends normalized events to S3

func (*S3Destination) SendEvents

func (d *S3Destination) SendEvents(parsedEventChannel chan *parsers.Result, errChan chan error)

SendEvents stores events in S3. It continuously reads events from parsedEventChannel, groups them in batches per log type and stores them in the appropriate S3 path. If the method encounters an error it writes an error to the errorChannel and continues until channel is closed (skipping events). The sendData() method is called as go routine to allow processing to continue and hide network latency.