encoding

README

logplex/encoding

What's this?

A set of libraries we use to parse messages, and to also publish these same syslog RFC5424 messages.

How to use?

We have 2 scanners available. If you're trying to build a logplex compatible ingress, you can use the regular scanner.

Scanner

func handler(w http.ResponseWriter, r *http.Request) {
	s := NewScanner(r.Body)

	for s.Scan() {
		log.Printf("%+v", scanner.Message())
	}

	if s.Err() != nil {
		log.Printf("err: %v", s.Err())
	}
}

DrainScanner

If the intent is to write an application which acts as a heroku drain, then using the DrainScanner is preferrable -- primarily because it doesn't require structured data.

func handler(w http.ResponseWriter, r *http.Request) {
	s := NewDrainScanner(r.Body)

	for s.Scan() {
		log.Printf("%+v", scanner.Message())
	}

	if s.Err() != nil {
		log.Printf("err: %v", s.Err())
	}
}

Documentation

Index

• Constants
• Variables
• func Encode(msg Message) ([]byte, error)
• type Encoder
  • func NewPlain(w io.Writer) Encoder
  • func NewSSE(w io.Writer) Encoder
• type Message
  • func Decode(raw []byte, hasStructuredData bool) (Message, error)
  • func (m Message) Size() (int, error)
  • func (m Message) WriteTo(w io.Writer) (int64, error)
• type Scanner
  • func NewDrainScanner(r io.ReadCloser) Scanner
  • func NewScanner(r io.Reader) Scanner

Constants

const FlexibleSyslogTimeFormat = "2006-01-02T15:04:05.999999Z07:00"

FlexibleSyslogTimeFormat accepts both 'Z' and TZ notation for event time.

const L15Error = "L15: Error displaying log lines. Please try again."

L15Error is the message returned with an L15 error

const MaxFrameLength = 10240

MaxFrameLength is the maximum message size to parse

const OptimalFrameLength = 1024

OptimalFrameLength is the initial buffer size for scanning

const SyslogTimeFormat = "2006-01-02T15:04:05.999999-07:00"

SyslogTimeFormat defines the exact time format used in our logs.

Variables

var ErrBadFrame = errors.New("bad frame")

ErrBadFrame is returned when the scanner cannot parse syslog message boundaries

var ErrInvalidMessage = errors.New("invalid message")

ErrInvalidMessage returned when trying to encode an invalid syslog message

var ErrInvalidStructuredData = errors.New("invalid structured data")

ErrInvalidStructuredData is returned when structure data has any value other than '-' (blank) We do not

Functions

func Encode

func Encode(msg Message) ([]byte, error)

Encode serializes a syslog message into their wire format ( octet-framed syslog ) Disabling RFC 5424 compliance is the default and needed due to https://github.com/heroku/logplex/issues/204

Types

type Encoder

type Encoder interface {
	Encode(msg Message) error
	KeepAlive() error
}

Encoder abstracts away how messages are written out

func NewPlain

func NewPlain(w io.Writer) Encoder

NewPlain creates a plain encoder. It dumps the log message directly without massaging it

func NewSSE

func NewSSE(w io.Writer) Encoder

NewSSE instantiates a new SSE encoder

type Message

type Message struct {
	Timestamp    time.Time
	Hostname     string
	Application  string
	Process      string
	ID           string
	Message      string
	Version      uint16
	Priority     uint8
	RFCCompliant bool
}

Message is a syslog message

func Decode

func Decode(raw []byte, hasStructuredData bool) (Message, error)

Decode converts a rfc5424 message to our model

func (Message) Size

func (m Message) Size() (int, error)

Size returns the message size in bytes, including the octet framing header

func (Message) WriteTo

func (m Message) WriteTo(w io.Writer) (int64, error)

WriteTo writes the message to a stream

type Scanner

type Scanner interface {
	Scan() bool
	Err() error
	Message() Message
}

Scanner is the general purpose primitive for parsing message bodies coming from log-shuttle, logfwd, logplex and all sorts of logging components.

func NewDrainScanner

func NewDrainScanner(r io.ReadCloser) Scanner

NewDrainScanner returns a scanner for use with drain endpoints. The primary difference is that it's lose and doesn't check for structured data.

func NewScanner

func NewScanner(r io.Reader) Scanner

NewScanner is a syslog octet frame stream parser