collector

Documentation

Overview

Package collector implements the LogDog Collector daemon's log parsing and registration logic.

The LogDog Collector is responsible for ingesting logs from the Butler sent via transport, stashing them in intermediate storage, and registering them with the LogDog Coordinator service.

Index

• Constants
• type Collector
  • func (c *Collector) Close()
  • func (c *Collector) Process(ctx context.Context, msg []byte) error

Constants

const (
	// DefaultMaxMessageWorkers is the default number of concurrent worker
	// goroutones to employ for a single message.
	DefaultMaxMessageWorkers = 4
)

Types

type Collector

type Collector struct {
	// Coordinator is used to interface with the Coordinator client.
	//
	// On production systems, this should wrapped with a caching client (see
	// the stateCache sub-package) to avoid overwhelming the server.
	Coordinator coordinator.Coordinator

	// Storage is the intermediate storage instance to use.
	Storage storage.Storage

	// StreamStateCacheExpire is the maximum amount of time that a cached stream
	// state entry is valid. If zero, DefaultStreamStateCacheExpire will be used.
	StreamStateCacheExpire time.Duration

	// MaxMessageWorkers is the maximum number of concurrent workers to employ
	// for any given message. If <= 0, DefaultMaxMessageWorkers will be applied.
	MaxMessageWorkers int
}

Collector is a stateful object responsible for ingesting LogDog logs, registering them with a Coordinator, and stowing them in short-term storage for streaming and processing.

A Collector's Close should be called when finished to release any internal resources.

func (*Collector) Close

func (c *Collector) Close()

Close releases any internal resources and blocks pending the completion of any outstanding operations. After Close, no new Process calls may be made.

func (*Collector) Process

func (c *Collector) Process(ctx context.Context, msg []byte) error

Process ingests an encoded ButlerLogBundle message, registering it with the LogDog Coordinator and stowing it in a temporary Storage for streaming retrieval.

If a transient error occurs during ingest, Process will return an error. If no error occurred, or if there was an error with the input data, no error will be returned.