write

Documentation

Index

• Constants
• Variables
• func ScanLines(data []byte, atEOF bool) (advance int, token []byte, err error)
• type Batcher
  • func (b *Batcher) Write(ctx context.Context, org, bucket platform.ID, r io.Reader) error
  • func (b *Batcher) WriteTo(ctx context.Context, filter platform.BucketFilter, r io.Reader) error

Constants

const (
	// DefaultMaxBytes is 500KB; this is typically 250 to 500 lines.
	DefaultMaxBytes = 500000
	// DefaultInterval will flush every 10 seconds.
	DefaultInterval = 10 * time.Second
)

Variables

var (
	// ErrLineTooLong is the error returned when reading a line that exceeds MaxLineLength.
	ErrLineTooLong = errors.New("batcher: line too long")
)

Functions

func ScanLines

func ScanLines(data []byte, atEOF bool) (advance int, token []byte, err error)

ScanLines is used in bufio.Scanner.Split to split lines of line protocol.

Types

type Batcher

type Batcher struct {
	MaxFlushBytes    int                   // MaxFlushBytes is the maximum number of bytes to buffer before flushing
	MaxFlushInterval time.Duration         // MaxFlushInterval is the maximum amount of time to wait before flushing
	MaxLineLength    int                   // MaxLineLength specifies the maximum length of a single line
	Service          platform.WriteService // Service receives batches flushed from Batcher.
}

Batcher batches line protocol for sends to output.

func (*Batcher) Write

func (b *Batcher) Write(ctx context.Context, org, bucket platform.ID, r io.Reader) error

Write reads r in batches and writes to a target specified by org and bucket.

func (*Batcher) WriteTo

func (b *Batcher) WriteTo(ctx context.Context, filter platform.BucketFilter, r io.Reader) error

WriteTo reads r in batches and writes to a target specified by filter.