log

package
v2.2.11+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 15, 2021 License: AGPL-3.0 Imports: 19 Imported by: 0

README

Log Service

A simple implementation of a log repository that receives all log messages via gRPC and store them in a bleve repository.

REST API

TODO

Documentation

Overview

Package log provides a persistence layer for json-formatted logs generated by the application.

It is intended to provide an out-of-the-box solution for storing and querying the logs, but should be replaced by more scalable solutions like ELK.

Index

Constants

View Source
const (
	MinRotationSize     = 68 * 1024
	DefaultRotationSize = int64(200 * 1024 * 1024)
)

Variables

This section is empty.

Functions

func BleveDeleteLogs added in v1.4.0

func BleveDeleteLogs(idx bleve.Index, str string) (int64, error)

BleveDeleteLogs queries the bleve index, based on the passed query string and deletes the results

func BleveDuplicateIndex added in v1.5.0

func BleveDuplicateIndex(from bleve.Index, inserts chan interface{}, logger func(string)) error

func BleveListLogs

func BleveListLogs(idx bleve.Index, str string, page int32, size int32) (chan log.ListLogResponse, error)

BleveListLogs queries the bleve index, based on the passed query string. It returns the results as a stream of log.ListLogResponse with the values of the indexed fields for each corresponding hit. Results are ordered by descending timestamp rather than by score.

func BlevePutLog

func BlevePutLog(idx bleve.Index, line map[string]string) error

BlevePutLog stores a new log msg in a bleve index. It expects a map[string]string retrieved from a deserialized proto log message.

func UnmarshallLogMsgFromDoc

func UnmarshallLogMsgFromDoc(doc *document.Document, msg *log.LogMessage)

UnmarshallLogMsgFromDoc populates the LogMessage from the passed bleve document.

func UnmarshallLogMsgFromFields

func UnmarshallLogMsgFromFields(m map[string]interface{}, msg *log.LogMessage)

Types

type IndexableLog

type IndexableLog struct {
	Nano int
	log.LogMessage
}

IndexableLog extends default log.LogMessage struct to add index specific methods

func MarshallLogMsg

func MarshallLogMsg(line map[string]string) (*IndexableLog, error)

MarshallLogMsg creates an IndexableLog object and populates the inner LogMessage with known fields of the passed JSON line.

type MessageRepository

type MessageRepository interface {
	PutLog(map[string]string) error
	ListLogs(string, int32, int32) (chan log.ListLogResponse, error)
	DeleteLogs(string) (int64, error)
	AggregatedLogs(string, string, int32) (chan log.TimeRangeResponse, error)
	Resync(logger *zap.Logger) error
	Truncate(max int64, logger *zap.Logger) error
}

MessageRepository exposes interface methods to manage the log messages provided by Pydio.

type SyslogServer

type SyslogServer struct {
	SearchIndex bleve.IndexAlias
	// contains filtered or unexported fields
}

SyslogServer is the syslog specific implementation of the Log server

func NewSyslogServer

func NewSyslogServer(indexPath string, mappingName string, rotationSize int64) (*SyslogServer, error)

NewSyslogServer creates and configures a default Bleve instance to store technical logs Setting rotationSize to -1 fully disables rotation

func (*SyslogServer) AggregatedLogs

func (s *SyslogServer) AggregatedLogs(msgId string, timeRangeType string, refTime int32) (chan log.TimeRangeResponse, error)

AggregatedLogs performs a faceted query in the syslog repository. UNIMPLEMENTED.

func (*SyslogServer) Close added in v1.5.0

func (s *SyslogServer) Close()

func (*SyslogServer) DeleteLogs added in v1.4.0

func (s *SyslogServer) DeleteLogs(query string) (int64, error)

DeleteLogs truncate logs based on a search query

func (*SyslogServer) ListLogs

func (s *SyslogServer) ListLogs(str string, page, size int32) (chan log.ListLogResponse, error)

ListLogs performs a query in the bleve index, based on the passed query string. It returns results as a stream of log.ListLogResponse for each corresponding hit. Results are ordered by descending timestamp rather than by score.

func (*SyslogServer) Open

func (s *SyslogServer) Open(indexPath string, mappingName string) error

func (*SyslogServer) PutLog

func (s *SyslogServer) PutLog(line map[string]string) error

PutLog adds a new LogMessage in the syslog index.

func (*SyslogServer) Resync added in v1.5.0

func (s *SyslogServer) Resync(logger *zap.Logger) error

Resync creates a copy of current index. It has been originally used for switching analyze format from bleve to scorch.

func (*SyslogServer) Truncate

func (s *SyslogServer) Truncate(max int64, logger *zap.Logger) error

Truncate gathers size of existing indexes, starting from last. When max is reached it starts deleting all previous indexes.

Directories

Path Synopsis
Package grpc provides a Pydio GRPC service for querying the logs
Package grpc provides a Pydio GRPC service for querying the logs
Package rest exposes a simple REST API for communicating with the GRPC package.
Package rest exposes a simple REST API for communicating with the GRPC package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL