p4dlog

README

go-libp4dlog

go-libp4dlog is a library for Go to parse Perforce p4d text logs.

P4D log files are written to P4LOG, or "p4d -L log". We would normally recommend configurables server=1 and track1 though you need to ensure your log file is regularly rotated.

For outline of how to setup P4LOG:

https://www.perforce.com/manuals/p4sag/Content/P4SAG/DB5-79706.html

P4D Log Analysis

See open source project:

• https://swarm.workshop.perforce.com/projects/perforce-software-log-analyzer

Also KB articles:

• https://community.perforce.com/s/article/2514
• https://community.perforce.com/s/article/2525

Output of this library

This library can output the results of log parsing as JSON (in future SQL statements for SQLite or MySQL).

It is used by:

• https://github.com/rcowham/p4dbeat - Custom Elastic Beat - consumes parsed log records and sends to Elastic stash
• https://github.com/rcowham/p4prometheus - consumes parsed log records and writes Prometheus metrics

Documentation

Overview

Package p4dlog parses Perforce Hexlix Core Server text logs (not structured logs).

These are logs created by p4d, as documented by:

https://community.perforce.com/s/article/2525

It assumes you have set configurable server=3 (or greater) You may also have decided to set track=1 to get more detailed usage of access to different tables.

See p4dlog_test.go for examples of log entries.

Index

• type Block
• type Command
  • func (c *Command) GetKey() string
  • func (c *Command) MarshalJSON() ([]byte, error)
  • func (c *Command) String() string
• type P4dFileParser
  • func NewP4dFileParser(logger *logrus.Logger) *P4dFileParser
  • func (fp *P4dFileParser) CmdsPendingCount() int
  • func (fp *P4dFileParser) LogParser(ctx context.Context, lines chan []byte, cmdchan chan Command)
  • func (fp *P4dFileParser) SetDebugMode()
  • func (fp *P4dFileParser) SetDurations(outputDuration, debugDuration time.Duration)
• type P4dParseOptions
• type Table

Types

type Block

type Block struct {
	// contains filtered or unexported fields
}

Block is a block of lines parsed from a file

type Command

type Command struct {
	ProcessKey     string    `json:"processKey"`
	Cmd            []byte    `json:"cmd"`
	Pid            int64     `json:"pid"`
	LineNo         int64     `json:"lineNo"`
	User           []byte    `json:"user"`
	Workspace      []byte    `json:"workspace"`
	StartTime      time.Time `json:"startTime"`
	EndTime        time.Time `json:"endTime"`
	ComputeLapse   float32   `json:"computeLapse"`
	CompletedLapse float32   `json:"completedLapse"`
	IP             []byte    `json:"ip"`
	App            []byte    `json:"app"`
	Args           []byte    `json:"args"`
	Running        int64     `json:"running"`
	UCpu           int64     `json:"uCpu"`
	SCpu           int64     `json:"sCpu"`
	DiskIn         int64     `json:"diskIn"`
	DiskOut        int64     `json:"diskOut"`
	IpcIn          int64     `json:"ipcIn"`
	IpcOut         int64     `json:"ipcOut"`
	MaxRss         int64     `json:"maxRss"`
	PageFaults     int64     `json:"pageFaults"`
	RpcMsgsIn      int64     `json:"rpcMsgsIn"`
	RpcMsgsOut     int64     `json:"rpcMsgsOut"`
	RpcSizeIn      int64     `json:"rpcSizeIn"`
	RpcSizeOut     int64     `json:"rpcSizeOut"`
	RpcHimarkFwd   int64     `json:"rpcHimarkFwd"`
	RpcHimarkRev   int64     `json:"rpcHimarkRev"`
	RpcSnd         float32   `json:"rpcSnd"`
	RpcRcv         float32   `json:"rpcRcv"`
	CmdError       bool      `json:"cmderror"`
	Tables         map[string]*Table
	// contains filtered or unexported fields
}

Command is a command found in the block

func (*Command) GetKey

func (c *Command) GetKey() string

GetKey - returns process key (handling duplicates)

func (*Command) MarshalJSON

func (c *Command) MarshalJSON() ([]byte, error)

MarshalJSON - handle time formatting

func (*Command) String

func (c *Command) String() string

type P4dFileParser

type P4dFileParser struct {
	// contains filtered or unexported fields
}

P4dFileParser - manages state

func NewP4dFileParser

func NewP4dFileParser(logger *logrus.Logger) *P4dFileParser

NewP4dFileParser - create and initialise properly

func (*P4dFileParser) CmdsPendingCount

func (fp *P4dFileParser) CmdsPendingCount() int

CmdsPendingCount - count of unmatched commands

func (*P4dFileParser) LogParser

func (fp *P4dFileParser) LogParser(ctx context.Context, lines chan []byte, cmdchan chan Command)

LogParser - interface to be run on a go routine - commands are returned on cmdchan

func (*P4dFileParser) SetDebugMode

func (fp *P4dFileParser) SetDebugMode()

SetDebugMode - turn on debugging - very verbose!

func (*P4dFileParser) SetDurations

func (fp *P4dFileParser) SetDurations(outputDuration, debugDuration time.Duration)

SetDurations - for debugging

type P4dParseOptions

type P4dParseOptions struct {
	File string
	// contains filtered or unexported fields
}

P4dParseOptions - Options for parsing - mainly for command line usage

type Table

type Table struct {
	TableName          string  `json:"tableName"`
	PagesIn            int64   `json:"pagesIn"`
	PagesOut           int64   `json:"pagesOut"`
	PagesCached        int64   `json:"pagesCached"`
	PagesSplitInternal int64   `json:"pagesSplitInternal"`
	PagesSplitLeaf     int64   `json:"pagesSplitLeaf"`
	ReadLocks          int64   `json:"readLocks"`
	WriteLocks         int64   `json:"writeLocks"`
	GetRows            int64   `json:"getRows"`
	PosRows            int64   `json:"posRows"`
	ScanRows           int64   `json:"scanRows"`
	PutRows            int64   `json:"putRows"`
	DelRows            int64   `json:"delRows"`
	TotalReadWait      int64   `json:"totalReadWait"`
	TotalReadHeld      int64   `json:"totalReadHeld"`
	TotalWriteWait     int64   `json:"totalWriteWait"`
	TotalWriteHeld     int64   `json:"totalWriteHeld"`
	MaxReadWait        int64   `json:"maxReadWait"`
	MaxReadHeld        int64   `json:"maxReadHeld"`
	MaxWriteWait       int64   `json:"maxWriteWait"`
	MaxWriteHeld       int64   `json:"maxWriteHeld"`
	PeekCount          int64   `json:"peekCount"`
	TotalPeekWait      int64   `json:"totalPeekWait"`
	TotalPeekHeld      int64   `json:"totalPeekHeld"`
	MaxPeekWait        int64   `json:"maxPeekWait"`
	MaxPeekHeld        int64   `json:"maxPeekHeld"`
	TriggerLapse       float32 `json:"triggerLapse"`
}

Table stores track information per table (part of Command)