logtrics

README

logtrics

logtrics provide a way to parse logs, to generate metrics, notify and more. It can read logs from multiple sources(console, UDP, TCP). It also provides interfaces through lua script to configure and customize your logging tricks :P

Configuration

sample

Usage:
  logtrics [flags]

Flags:
      --buffer.size int         go channel default buffer size
  -c, --config string           config file path (default "/etc/logtrics/logtrics.toml")
      --graphite.debug          if enabled metrics will be logged
      --graphite.host string    graphite server host (default "127.0.0.1")
      --graphite.interval int   interval in secs (default 30)
      --graphite.port int       graphite server port (default 2024)
  -h, --help                    help for logtrics
      --logging.level string    logging level (default "info")
      --logging.type string     logging type, choices are "syslog", "console" (default "console")
  -m, --modes strings           run modes, choices are "console", "udp", "tcp"'
  -d, --script.dir string       lua scripts directory (default "/etc/logtrics/scripts/")
  -f, --script.file string      lua script file path
      --tcp.host string         tcp server listening host (default "127.0.0.1")
      --tcp.port int            tcp server listening port (default 4003)
      --udp.host string         udp server listening host (default "127.0.0.1")
      --udp.port int            udp server listening port (default 4002)
  -v, --version                 version for logtrics

Modes

logtrics supports multiple mode to receive log line.

• console - Mainly for debugging scripts
• UDP/TCP - Receives logs using UDP/TCP socket. Mainly to be used with rsyslog omfwd
• filetail - Receives logs by tailing log file. (TODO)

Console

In this mode, the log lines can be provided to the console prompt to debug the scripts.

logtrics -m console -f examples/scripts/logtrics.lua --logging.level debug

UDP

In this mode, the log lines can be read from the UDP socket

logtrics -m udp -f examples/scripts/logtrics.lua --logging.level debug --udp.port 4002 --udp.host localhost

send logs using echo "hello \"World\"" | nc -cu localhost 4002

TCP

In this mode, the log lines can be read from the TCP socket

logtrics -m tcp -f examples/scripts/logtrics.lua --logging.level debug --tcp.port 4003 --tcp.host localhost

send logs using echo "hello \"World\"" | nc -c localhost 4003

Lua Script

sample

logtrics {
	name = "logtrics-example",
	parser = {
		type = "re2",
		-- expression for `hello "World"`. extracting word hello
		expression = 'hello "(?P<first>[a-zA-z0-9]+)"',
	},
	handler = function(fields)
		info("fields are %v" , fields)
		-- graphite().counter("demo.counter.inc.value").inc(value)
		-- graphite().counter("demo.counter.dec.value").dec(value)
		-- graphite().timer("demo.timer.value").update(value)
		-- graphite().gauge("demo.gauge.value").update(value)
		-- graphite().meter("demo.meter.value").mark(value)
		end,
}

TODO

Documentation

Index

• Constants
• type Application
  • func NewApplication(conf *config.Configuration, readers ...reader.LogReader) (*Application, error)
  • func (app *Application) Run(ctx context.Context) error
  • func (app *Application) RunAsync(ctx context.Context) error
• type Logtric
  • func NewLogtric(script string, conf *config.Configuration, state *lua.LState, ...) (*Logtric, error)
  • func (l *Logtric) LAPIDebug(state *lua.LState) int
  • func (l *Logtric) LAPIError(state *lua.LState) int
  • func (l *Logtric) LAPIFatal(state *lua.LState) int
  • func (l *Logtric) LAPIGraphite(state *lua.LState) int
  • func (l *Logtric) LAPIInfo(state *lua.LState) int
  • func (l *Logtric) LAPITrace(state *lua.LState) int
  • func (l *Logtric) LAPIWarn(state *lua.LState) int
  • func (l *Logtric) Run(ctx context.Context, event reader.LogEvent) error
• type Parser
  • func NewParser(table *lua.LTable) (Parser, error)
• type RE2
  • func (p *RE2) FindSubStrings(s string) (map[string]string, bool)
• type Script
  • func NewScript(path string, conf *config.Configuration) (*Script, error)
  • func (s *Script) LAPILogtric(state *lua.LState) int
  • func (s *Script) Run(ctx context.Context, event reader.LogEvent)
  • func (s *Script) RunAsync(ctx context.Context, c <-chan reader.LogEvent)

Constants

const Version = "1.0.0"

Version defines the version of the application

Types

type Application

type Application struct {
	// contains filtered or unexported fields
}

Application represents this application it stores all the application states and maintains the runtime

func NewApplication

func NewApplication(conf *config.Configuration, readers ...reader.LogReader) (*Application, error)

NewApplication returns a new Application instance

func (*Application) Run

func (app *Application) Run(ctx context.Context) error

Run runs the application returns error in case of any failure note: this is a blocking call.

func (*Application) RunAsync

func (app *Application) RunAsync(ctx context.Context) error

RunAsync runs the application in async mode. returns error in case of any failure note: this is a blocking call.

type Logtric

type Logtric struct {
	// contains filtered or unexported fields
}

Logtric represents the logtrics instance configured in lua it stores the lua script states and provides runtime bindings to lua

func NewLogtric

func NewLogtric(script string, conf *config.Configuration, state *lua.LState, table *lua.LTable) (*Logtric, error)

NewLogtric returns a new instance of Logtric

func (*Logtric) LAPIDebug

func (l *Logtric) LAPIDebug(state *lua.LState) int

LAPIDebug represents the lua binding for debug() function call

func (*Logtric) LAPIError

func (l *Logtric) LAPIError(state *lua.LState) int

LAPIError represent the lua binding for error() function call

func (*Logtric) LAPIFatal

func (l *Logtric) LAPIFatal(state *lua.LState) int

LAPIFatal represent the lua binding for fatal() function call

func (*Logtric) LAPIGraphite

func (l *Logtric) LAPIGraphite(state *lua.LState) int

LAPIGraphite is represents the lua binding for graphite() api call

func (*Logtric) LAPIInfo

func (l *Logtric) LAPIInfo(state *lua.LState) int

LAPIInfo represents the lua binding for info() function call

func (*Logtric) LAPITrace

func (l *Logtric) LAPITrace(state *lua.LState) int

LAPITrace represent the lua binding for fatal() function call

func (*Logtric) LAPIWarn

func (l *Logtric) LAPIWarn(state *lua.LState) int

LAPIWarn represent the lua binding for error() function call

func (*Logtric) Run

func (l *Logtric) Run(ctx context.Context, event reader.LogEvent) error

Run runs the Logtric instance

type Parser

type Parser interface {
	FindSubStrings(s string) (map[string]string, bool)
}

Parser represents the implementation of string parsing

func NewParser

func NewParser(table *lua.LTable) (Parser, error)

NewParser returns a new parser instance

type RE2

type RE2 struct {
	// contains filtered or unexported fields
}

RE2 represents RE2 expression parser

func (*RE2) FindSubStrings

func (p *RE2) FindSubStrings(s string) (map[string]string, bool)

FindSubStrings extracts the sub strings from the string

type Script

type Script struct {
	Path string
	// contains filtered or unexported fields
}

Script represents the logtrics lua script

func NewScript

func NewScript(path string, conf *config.Configuration) (*Script, error)

NewScript returns a new Script instance which represents a lua script file

func (*Script) LAPILogtric

func (s *Script) LAPILogtric(state *lua.LState) int

LAPILogtric represents lua binding for logtric initialization

func (*Script) Run

func (s *Script) Run(ctx context.Context, event reader.LogEvent)

Run runs the script This is non blocking call

func (*Script) RunAsync

func (s *Script) RunAsync(ctx context.Context, c <-chan reader.LogEvent)

RunAsync runs the script in async mode It consumes the log event from the channel note: this is a blocking call