log

README

Log Service

A simple implementation of a log repository that receives all log messages via gRPC and store them in a bleve repository.

Documentation

Overview

Package log provides a persistence layer for json-formatted logs generated by the application.

It is intended to provide an out-of-the-box solution for storing and querying the logs, but should be replaced by more scalable solutions like ELK.

Index

• Variables
• func Migrate(mainCtx, fromCtx, toCtx context.Context, dryRun bool, ...) (map[string]int, error)
• type BaseCodec
  • func (b *BaseCodec) Marshal(input interface{}) (interface{}, error)
• type IndexRepository
  • func (s *IndexRepository) AggregatedLogs(_ context.Context, _ string, _ string, _ int32) (chan log.TimeRangeResponse, error)
  • func (s *IndexRepository) Close(ctx context.Context) error
  • func (s *IndexRepository) DeleteLogs(ctx context.Context, query string) (int64, error)
  • func (s *IndexRepository) Init(ctx context.Context, conf configx.Values) error
  • func (s *IndexRepository) ListLogs(ctx context.Context, str string, page int32, size int32) (chan log.ListLogResponse, error)
  • func (s *IndexRepository) NewBatch(ctx context.Context, options ...indexer.BatchOption) (indexer.Batch, error)
  • func (s *IndexRepository) PutLog(ctx context.Context, line *log.Log) error
  • func (s *IndexRepository) Resync(ctx context.Context, logger log2.ZapLogger) error
  • func (s *IndexRepository) Stats(ctx context.Context) map[string]interface{}
  • func (s *IndexRepository) Truncate(ctx context.Context, max int64, logger log2.ZapLogger) error
• type IndexableLog
  • func (*IndexableLog) BleveType() string
• type MessageRepository
  • func NewIndexRepository(idx indexer.Indexer) MessageRepository

Variables

var Drivers = service.StorageDrivers{}

Functions

func Migrate

func Migrate(mainCtx, fromCtx, toCtx context.Context, dryRun bool, status chan service.MigratorStatus) (map[string]int, error)

Types

type BaseCodec

type BaseCodec struct{}

func (*BaseCodec) Marshal

func (b *BaseCodec) Marshal(input interface{}) (interface{}, error)

type IndexRepository

type IndexRepository struct {
	indexer.Indexer
}

func (*IndexRepository) AggregatedLogs

func (s *IndexRepository) AggregatedLogs(_ context.Context, _ string, _ string, _ int32) (chan log.TimeRangeResponse, error)

AggregatedLogs performs a faceted query in the syslog repository. UNIMPLEMENTED.

func (*IndexRepository) Close

func (s *IndexRepository) Close(ctx context.Context) error

func (*IndexRepository) DeleteLogs

func (s *IndexRepository) DeleteLogs(ctx context.Context, query string) (int64, error)

DeleteLogs truncate logs based on a search query

func (*IndexRepository) Init

func (s *IndexRepository) Init(ctx context.Context, conf configx.Values) error

func (*IndexRepository) ListLogs

func (s *IndexRepository) ListLogs(ctx context.Context, str string, page int32, size int32) (chan log.ListLogResponse, error)

ListLogs performs a query in the bleve index, based on the passed query string. It returns results as a stream of log.ListLogResponse for each corresponding hit. Results are ordered by descending timestamp rather than by score.

func (*IndexRepository) NewBatch

func (s *IndexRepository) NewBatch(ctx context.Context, options ...indexer.BatchOption) (indexer.Batch, error)

func (*IndexRepository) PutLog

func (s *IndexRepository) PutLog(ctx context.Context, line *log.Log) error

PutLog adds a new LogMessage in the syslog index.

func (*IndexRepository) Resync

func (s *IndexRepository) Resync(ctx context.Context, logger log2.ZapLogger) error

func (*IndexRepository) Stats

func (s *IndexRepository) Stats(ctx context.Context) map[string]interface{}

func (*IndexRepository) Truncate

func (s *IndexRepository) Truncate(ctx context.Context, max int64, logger log2.ZapLogger) error

type IndexableLog

type IndexableLog struct {
	Nano            int `bson:"nano"`
	*log.LogMessage `bson:"inline"`
}

IndexableLog extends default log.LogMessage struct to add index specific methods

func (*IndexableLog) BleveType

func (*IndexableLog) BleveType() string

BleveType is interpreted by bleve indexer as the mapping name

type MessageRepository

type MessageRepository interface {
	PutLog(context.Context, *log.Log) error
	ListLogs(context.Context, string, int32, int32) (chan log.ListLogResponse, error)
	DeleteLogs(context.Context, string) (int64, error)
	AggregatedLogs(context.Context, string, string, int32) (chan log.TimeRangeResponse, error)
	Close(context.Context) error
	Resync(context.Context, log2.ZapLogger) error
	Truncate(context.Context, int64, log2.ZapLogger) error
	NewBatch(ctx context.Context, options ...indexer.BatchOption) (indexer.Batch, error)
}

MessageRepository exposes interface methods to manage the log messages provided by Pydio.

func NewIndexRepository

func NewIndexRepository(idx indexer.Indexer) MessageRepository