kinesis

Documentation

Overview

Package kinesis contains the OutputPlugin which sends log records to Kinesis Stream

Index

• Constants
• type CompressionType
• type CompressorFunc
• type OutputPlugin
  • func NewOutputPlugin(...) (*OutputPlugin, error)
  • func (outputPlugin *OutputPlugin) AddRecord(records *[]*kinesis.PutRecordsRequestEntry, record map[interface{}]interface{}, ...) int
  • func (outputPlugin *OutputPlugin) Flush(records *[]*kinesis.PutRecordsRequestEntry) int
  • func (outputPlugin *OutputPlugin) FlushAggregatedRecords(records *[]*kinesis.PutRecordsRequestEntry) int
  • func (outputPlugin *OutputPlugin) FlushConcurrent(count int, records []*kinesis.PutRecordsRequestEntry) int
  • func (outputPlugin *OutputPlugin) FlushWithRetries(count int, records []*kinesis.PutRecordsRequestEntry)
  • func (outputPlugin *OutputPlugin) IsAggregate() bool
• type PutRecordsClient

Constants

const (
	// CompressionNone disables compression
	CompressionNone CompressionType = "none"
	// CompressionZlib enables zlib compression
	CompressionZlib = "zlib"
	// CompressionGzip enables gzip compression
	CompressionGzip = "gzip"
)

Types

type CompressionType

type CompressionType string

CompressionType indicates the type of compression to apply to each record

type CompressorFunc

type CompressorFunc func([]byte) ([]byte, error)

CompressorFunc is a function that compresses a byte slice

type OutputPlugin

type OutputPlugin struct {
	PluginID int

	Concurrency int
	// contains filtered or unexported fields
}

OutputPlugin sends log records to kinesis

func NewOutputPlugin

func NewOutputPlugin(region, stream, dataKeys, partitionKey, roleARN, kinesisEndpoint, stsEndpoint, timeKey, timeFmt, logKey, replaceDots string, concurrency, retryLimit int, isAggregate, appendNewline bool, compression CompressionType, pluginID int, httpRequestTimeout time.Duration, aggregationMaximumRecordSize, skipAggregationRecordSize *int) (*OutputPlugin, error)

NewOutputPlugin creates an OutputPlugin object

func (*OutputPlugin) AddRecord

func (outputPlugin *OutputPlugin) AddRecord(records *[]*kinesis.PutRecordsRequestEntry, record map[interface{}]interface{}, timeStamp *time.Time) int

AddRecord accepts a record and adds it to the buffer the return value is one of: FLB_OK FLB_RETRY FLB_ERROR

func (*OutputPlugin) Flush

func (outputPlugin *OutputPlugin) Flush(records *[]*kinesis.PutRecordsRequestEntry) int

Flush sends the current buffer of log records Returns FLB_OK, FLB_RETRY, FLB_ERROR

func (*OutputPlugin) FlushAggregatedRecords

func (outputPlugin *OutputPlugin) FlushAggregatedRecords(records *[]*kinesis.PutRecordsRequestEntry) int

FlushAggregatedRecords must be called after Returns FLB_OK, FLB_RETRY, FLB_ERROR

func (*OutputPlugin) FlushConcurrent

func (outputPlugin *OutputPlugin) FlushConcurrent(count int, records []*kinesis.PutRecordsRequestEntry) int

FlushConcurrent sends the current buffer of log records in a goroutine with retries Returns FLB_OK, FLB_RETRY Will return FLB_RETRY if the limit of concurrency has been reached

func (*OutputPlugin) FlushWithRetries

func (outputPlugin *OutputPlugin) FlushWithRetries(count int, records []*kinesis.PutRecordsRequestEntry)

FlushWithRetries sends the current buffer of log records, with retries

func (*OutputPlugin) IsAggregate

func (outputPlugin *OutputPlugin) IsAggregate() bool

IsAggregate indicates if this instance of the plugin has KCL aggregation enabled.

type PutRecordsClient

type PutRecordsClient interface {
	PutRecords(input *kinesis.PutRecordsInput) (*kinesis.PutRecordsOutput, error)
}

PutRecordsClient contains the kinesis PutRecords method call