Documentation ¶
Index ¶
Constants ¶
View Source
const ( LogData = "LogData" RuleOutput = "RuleOutput" )
View Source
const ( OpLogNamespace = "Panther" // FIXME: move "up" in the stack OpLogComponent = "LogProcessor" OpLogServiceDim = "Service" )
labels for oplog
Variables ¶
View Source
var ( OpLogManager = oplog.NewManager(OpLogNamespace, OpLogComponent) OpLogLambdaServiceDim = zap.String(OpLogServiceDim, "lambda") OpLogS3ServiceDim = zap.String(OpLogServiceDim, "s3") OpLogSNSServiceDim = zap.String(OpLogServiceDim, "sns") OpLogProcessorServiceDim = zap.String(OpLogServiceDim, "processor") OpLogGlueServiceDim = zap.String(OpLogServiceDim, "glue") )
View Source
var Session = session.Must(session.NewSession(aws.NewConfig().WithMaxRetries(10)))
Session AWS Session that can be used by components of the system Setting Max Retries to a higher number - we'd like to retry several times when reading from S3/pushing to Firehose before failing.
Functions ¶
This section is empty.
Types ¶
type DataStream ¶
type DataStream struct { Reader io.Reader Hints DataStreamHints // The log type if known // If it is nil, it means the log type hasn't been identified yet LogType *string }
DataStream represents a data stream that read by the processor
type DataStreamHints ¶
type DataStreamHints struct {
S3 *S3DataStreamHints // if nil, no hint
}
Used in a DataStream as meta data to describe the data
type ParsedEvent ¶
type ParsedEvent struct { Event interface{} `json:"event"` LogType string `json:"logType"` }
ParsedEvent contains a single event that has already been processed
type S3DataStreamHints ¶
Used in a DataStreamHints as meta data to describe the S3 object backing the stream
type S3Notification ¶
type S3Notification struct { // S3Bucket is name of the S3 Bucket where data is available S3Bucket *string `json:"s3Bucket"` // S3ObjectKey is the key of the S3 object that contains the new data S3ObjectKey *string `json:"s3ObjectKey"` // Events is the number of events in the S3 object Events *int `json:"events"` // Bytes is the uncompressed size in bytes of the S3 object Bytes *int `json:"bytes"` // Type is the type of data available in the S3 object (LogData,RuleOutput) Type *string `json:"type"` // ID is an identified for the data in the S3 object. In case of LogData this will be // the Log Type, in case of RuleOutput data this will be the RuleID ID *string `json:"id"` }
S3Notification is sent when new data is available in S3
Click to show internal directories.
Click to hide internal directories.