parsers

package
v1.13.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 30, 2020 License: AGPL-3.0 Imports: 18 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	PantherFieldPrefix = "p_"
)

Variables

View Source 
var Validator = NewValidator()

Validator can be used to validate schemas of log fields

Functions

func AppendAnyString added in v0.2.0 

func AppendAnyString(any *PantherAnyString, values ...string)

func CsvStringToArray added in v0.2.0 

func CsvStringToArray(value string) []string

func CsvStringToFloat64Pointer added in v0.2.0 

func CsvStringToFloat64Pointer(value string) *float64

func CsvStringToInt16Pointer added in v0.2.0 

func CsvStringToInt16Pointer(value string) *int16

func CsvStringToIntPointer added in v0.2.0 

func CsvStringToIntPointer(value string) *int

func CsvStringToPointer added in v0.2.0 

func CsvStringToPointer(value string) *string

func LooksLikeCSV added in v1.5.0 

func LooksLikeCSV(log string) bool

Returns true if log looks like a CSV log. It can be used to fail fast for logs that are not CSV

func NewValidator added in v1.7.0 

func NewValidator() *validator.Validate

NewValidator creates a validator.Validate instance that knows how to handle the types used in panther logs.

Types

type Factory added in v1.5.1 

type Factory interface {
	NewParser(params interface{}) (Interface, error)
}

Factory creates new parser instances. The params argument defines parameters for a parser.

func AdapterFactory added in v1.5.1 

func AdapterFactory(parser LogParser) Factory

AdapterFactory returns a parsers.Factory from a parsers.Parser This is used to ease transition to the new parsers.Interface for parsers based on parsers.PantherLog

type FactoryFunc added in v1.7.0 

type FactoryFunc func(params interface{}) (Interface, error)

FactoryFunc is a callback parser factory

func (FactoryFunc) NewParser added in v1.7.0 

func (ff FactoryFunc) NewParser(params interface{}) (Interface, error)

NewParser implements Factory interface

type Interface added in v1.5.1 

type Interface interface {
	ParseLog(log string) ([]*Result, error)
}

Interface is the interface to be used for log parsers.

func NewAdapter added in v1.5.1 

func NewAdapter(parser LogParser) Interface

NewAdapter creates a pantherlog.LogParser from a parsers.Parser

type JSONParserFactory added in v1.7.0 

type JSONParserFactory struct {
	LogType        string
	NewEvent       func() interface{}
	JSON           jsoniter.API
	Validate       func(event interface{}) error
	ReadBufferSize int
	NextRowID      func() string
	Now            func() time.Time
}

func (*JSONParserFactory) NewParser added in v1.7.0 

func (f *JSONParserFactory) NewParser(_ interface{}) (Interface, error)

type LogParser 

type LogParser interface {
	// LogType returns the log type supported by this parser
	LogType() string

	// Parse attempts to parse the provided log line
	// If the provided log is not of the supported type the method returns nil and an error
	Parse(log string) ([]*PantherLog, error)

	// New returns a new instance of the log parser, used like a factory method for stateful parsers
	New() LogParser
}

LogParser represents a parser for a supported log type NOTE: We will be transitioning parsers to the `parsers.Interface` interface. Until all parsers are converted to the new interface the `AdapterFactory()` helper should be used when registering a new log type to a `logtypes.Registry`

type PantherAnyString added in v0.2.0 

type PantherAnyString struct {
	// contains filtered or unexported fields
}

func NewPantherAnyString added in v0.2.0 

func NewPantherAnyString() *PantherAnyString

func (*PantherAnyString) MarshalJSON added in v0.2.0 

func (any *PantherAnyString) MarshalJSON() ([]byte, error)

func (*PantherAnyString) UnmarshalJSON added in v0.2.0 

func (any *PantherAnyString) UnmarshalJSON(jsonBytes []byte) error

type PantherLog added in v0.2.0 

type PantherLog struct {

	//  required
	PantherLogType     *string            `json:"p_log_type,omitempty" validate:"required" description:"Panther added field with type of log"`
	PantherRowID       *string            `json:"p_row_id,omitempty" validate:"required" description:"Panther added field with unique id (within table)"`
	PantherEventTime   *timestamp.RFC3339 `json:"p_event_time,omitempty" validate:"required" description:"Panther added standardize event time (UTC)"`
	PantherParseTime   *timestamp.RFC3339 `json:"p_parse_time,omitempty" validate:"required" description:"Panther added standardize log parse time (UTC)"`
	PantherSourceID    *string            `json:"p_source_id,omitempty" description:"Panther added field with the source id"`
	PantherSourceLabel *string            `json:"p_source_label,omitempty" description:"Panther added field with the source label"`

	// optional (any)
	PantherAnyIPAddresses  *PantherAnyString `json:"p_any_ip_addresses,omitempty" description:"Panther added field with collection of ip addresses associated with the row"`
	PantherAnyDomainNames  *PantherAnyString `json:"p_any_domain_names,omitempty" description:"Panther added field with collection of domain names associated with the row"`
	PantherAnySHA1Hashes   *PantherAnyString `json:"p_any_sha1_hashes,omitempty" description:"Panther added field with collection of SHA1 hashes associated with the row"`
	PantherAnyMD5Hashes    *PantherAnyString `json:"p_any_md5_hashes,omitempty" description:"Panther added field with collection of MD5 hashes associated with the row"`
	PantherAnySHA256Hashes *PantherAnyString `` /* 144-byte string literal not displayed */
	// contains filtered or unexported fields
}

All log parsers should extend from this to get standardized fields (all prefixed with 'p_' as JSON for uniqueness) NOTE: It is VERY important that fields are added to END of the structure to avoid needed to re-build existing Glue partitions. 

See https://github.com/awsdocs/amazon-athena-user-guide/blob/master/doc_source/updates-and-partitions.md

nolint(lll)

func (*PantherLog) AppendAnyDomainNamePtrs added in v0.2.0 

func (pl *PantherLog) AppendAnyDomainNamePtrs(values ...*string)

func (*PantherLog) AppendAnyDomainNames added in v0.2.0 

func (pl *PantherLog) AppendAnyDomainNames(values ...string)

func (*PantherLog) AppendAnyIPAddress added in v1.0.1 

func (pl *PantherLog) AppendAnyIPAddress(value string) bool

func (*PantherLog) AppendAnyIPAddressInField added in v1.1.0 

func (pl *PantherLog) AppendAnyIPAddressInField(value string) bool

AppendAnyIPAddressInField extracts all IPs from the value using a regexp

func (*PantherLog) AppendAnyIPAddressInFieldPtr added in v1.1.0 

func (pl *PantherLog) AppendAnyIPAddressInFieldPtr(value *string) bool

AppendAnyIPAddressInFieldPtr makes sure the value passed is not nil before passing into AppendAnyIPAddressInField

func (*PantherLog) AppendAnyIPAddressPtr added in v1.0.1 

func (pl *PantherLog) AppendAnyIPAddressPtr(value *string) bool

AppendAnyIPAddressPtr returns true if the IP address was successfully appended, otherwise false if the value was not an IP

func (*PantherLog) AppendAnyMD5HashPtrs added in v0.3.0 

func (pl *PantherLog) AppendAnyMD5HashPtrs(values ...*string)

func (*PantherLog) AppendAnyMD5Hashes added in v0.3.0 

func (pl *PantherLog) AppendAnyMD5Hashes(values ...string)

func (*PantherLog) AppendAnySHA1HashPtrs added in v0.3.0 

func (pl *PantherLog) AppendAnySHA1HashPtrs(values ...*string)

func (*PantherLog) AppendAnySHA1Hashes added in v0.3.0 

func (pl *PantherLog) AppendAnySHA1Hashes(values ...string)

func (*PantherLog) AppendAnySHA256Hashes added in v1.2.0 

func (pl *PantherLog) AppendAnySHA256Hashes(values ...string)

func (*PantherLog) AppendAnySHA256HashesPtr added in v1.2.0 

func (pl *PantherLog) AppendAnySHA256HashesPtr(values ...*string)

func (*PantherLog) Event added in v1.0.0 

func (pl *PantherLog) Event() interface{}

Event returns event data, used when composed

func (*PantherLog) Log added in v1.0.0 

func (pl *PantherLog) Log() *PantherLog

Log returns pointer to self, used when composed

func (*PantherLog) Logs added in v1.0.0 

func (pl *PantherLog) Logs() []*PantherLog

Logs returns a slice with pointer to self, used when composed

func (*PantherLog) Result added in v1.5.1 

func (pl *PantherLog) Result() *Result

Result converts a PantherLog to Result

func (*PantherLog) Results added in v1.5.1 

func (pl *PantherLog) Results() ([]*Result, error)

Results converts a PantherLog to a slice of results

func (*PantherLog) SetCoreFields added in v0.2.0 

func (pl *PantherLog) SetCoreFields(logType string, eventTime *timestamp.RFC3339, event interface{})

func (*PantherLog) SetEvent added in v1.0.0 

func (pl *PantherLog) SetEvent(event interface{})

SetEvent set event data, used for testing

func (*PantherLog) SetPantherSource added in v1.8.0 

func (pl *PantherLog) SetPantherSource(id, label string)

type PantherSourceSetter added in v1.8.0 

type PantherSourceSetter interface {
	SetPantherSource(id, label string)
}

type Result added in v1.5.1 

type Result = pantherlog.Result

Result is the result of parsing a log event. It is an alias of `pantherlog.Result` to help with the refactoring.

func ToResults added in v1.5.1 

func ToResults(logs []*PantherLog, err error) ([]*Result, error)

Source Files

View all Source files

Directories

Path Synopsis
Package apachelogs contains parsers for logs of the Apache HTTP Server
 Package apachelogs contains parsers for logs of the Apache HTTP Server
Package awslogs defines parsers and log types for AWS logs.
 Package awslogs defines parsers and log types for AWS logs.
Package gcplogs has log parsers for Google Cloud Platform
 Package gcplogs has log parsers for Google Cloud Platform
Package gitlablogs parses GitLab JSON logs.
 Package gitlablogs parses GitLab JSON logs.
Package juniperlogs provides parsers for Juniper logs
 Package juniperlogs provides parsers for Juniper logs
Package nginxlogs provides parsers for NGINX server logs
 Package nginxlogs provides parsers for NGINX server logs
Package sysloglogs provides parsers for syslog messages.
 Package sysloglogs provides parsers for syslog messages.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.