common

package
v5.0.0-alpha5+incompat... Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 3, 2016 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	EventMetadataKey = "_event_metadata"
	FieldsKey        = "fields"
	TagsKey          = "tags"
)
View Source 
const (
	OK_STATUS           = "OK"
	ERROR_STATUS        = "Error"
	SERVER_ERROR_STATUS = "Server Error"
	CLIENT_ERROR_STATUS = "Client Error"
)

standardized status values

View Source 
const MaxIpPortTupleRawSize = 16 + 16 + 2 + 2
View Source 
const MaxTcpTupleRawSize = 16 + 16 + 2 + 2 + 4
View Source 
const TsLayout = "2006-01-02T15:04:05.000Z"

TsLayout is the layout to be used in the timestamp marshaling/unmarshaling everywhere. The timezone must always be UTC.

Variables

View Source 
var ErrorFieldsIsNotMapStr = errors.New("the value stored in fields is not a MapStr")
View Source 
var ErrorTagsIsNotStringArray = errors.New("the value stored in tags is not a []string")

Functions

func AddTags 

func AddTags(ms MapStr, tags []string) error

AddTag appends a tag to the tags field of ms. If the tags field does not exist then it will be created. If the tags field exists and is not a []string then an error will be returned. It does not deduplicate the list of tags.

func Bytes_Htohl 

func Bytes_Htohl(b []byte) uint32

func Bytes_Ntohl 

func Bytes_Ntohl(b []byte) uint32

func Bytes_Ntohll 

func Bytes_Ntohll(b []byte) uint64

func Bytes_Ntohs 

func Bytes_Ntohs(b []byte) uint16

func DumpInCSVFormat 

func DumpInCSVFormat(fields []string, rows [][]string) string

DumpInCSVFormat takes a set of fields and rows and returns a string representing the CSV representation for the fields and rows.

func Ipv4_Ntoa 

func Ipv4_Ntoa(ip uint32) string

Ipv4_Ntoa transforms an IP4 address in it's dotted notation

func IsLoopback 

func IsLoopback(ip_str string) (bool, error)

IsLoopback check if a particular IP notation corresponds to a loopback interface.

func LoadGeoIPData 

func LoadGeoIPData(config Geoip) *libgeo.GeoIP

func LocalIpAddrs 

func LocalIpAddrs() ([]net.IP, error)

LocalIpAddrs finds the IP addresses of the hosts on which the shipper currently runs on.

func LocalIpAddrsAsStrings 

func LocalIpAddrsAsStrings(include_loopbacks bool) ([]string, error)

LocalIpAddrsAsStrings finds the IP addresses of the hosts on which the shipper currently runs on and returns them as an array of strings.

func MergeFields 

func MergeFields(ms, fields MapStr, underRoot bool) error

MergeFields merges the top-level keys and values in each source hash (it does not perform a deep merge). If the same key exists in both, the value in fields takes precedence. If underRoot is true then the contents of the fields MapStr is merged with the value of the 'fields' key in ms.

An error is returned if underRoot is true and the value of ms.fields is not a MapStr.

func NewFlagOverwrite 

func NewFlagOverwrite(
	set *flag.FlagSet,
	config *Config,
	name, path, def, usage string,
) *string

func ReadString 

func ReadString(s []byte) (string, error)

ReadString extracts the first null terminated string from a slice of bytes.

Types

type Backoff 

type Backoff struct {
	// contains filtered or unexported fields
}

A Backoff waits on errors with exponential backoff (limited by maximum backoff). Resetting Backoff will reset the next sleep timer to the initial backoff duration.

func NewBackoff 

func NewBackoff(done <-chan struct{}, init, max time.Duration) *Backoff

func (*Backoff) Reset 

func (b *Backoff) Reset()

func (*Backoff) TryWaitOnError 

func (b *Backoff) TryWaitOnError(failTS time.Time, err error) bool

func (*Backoff) Wait 

func (b *Backoff) Wait() bool

func (*Backoff) WaitOnError 

func (b *Backoff) WaitOnError(err error) bool

type Cache 

type Cache struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

Cache is a semi-persistent mapping of keys to values. Elements added to the cache are store until they are explicitly deleted or are expired due time- based eviction based on last access time.

Expired elements are not visible through classes methods, but they do remain stored in the cache until CleanUp() is invoked. Therefore CleanUp() must be invoked periodically to prevent the cache from becoming a memory leak. If you want to start a goroutine to perform periodic clean-up then see StartJanitor().

Cache does not support storing nil values. Any attempt to put nil into the cache will cause a panic.

func NewCache 

func NewCache(d time.Duration, initialSize int) *Cache

NewCache creates and returns a new Cache. d is the length of time after last access that cache elements expire. initialSize is the initial allocation size used for the Cache's underlying map.

func NewCacheWithRemovalListener 

func NewCacheWithRemovalListener(d time.Duration, initialSize int, l RemovalListener) *Cache

NewCacheWithRemovalListener creates and returns a new Cache and register a RemovalListener callback function. d is the length of time after last access that cache elements expire. initialSize is the initial allocation size used for the Cache's underlying map. l is the callback function that will be invoked when cache elements are removed from the map on CleanUp.

func (*Cache) CleanUp 

func (c *Cache) CleanUp() int

CleanUp performs maintenance on the cache by removing expired elements from the cache. If a RemoveListener is registered it will be invoked for each element that is removed during this clean up operation. The RemovalListener is invoked on the caller's goroutine.

func (*Cache) Delete 

func (c *Cache) Delete(k Key) Value

Delete a key from the map and return the value or nil if the key does not exist. The RemovalListener is not notified for explicit deletions.

func (*Cache) Entries 

func (c *Cache) Entries() map[Key]Value

Entries returns a shallow copy of the non-expired elements in the cache.

func (*Cache) Get 

func (c *Cache) Get(k Key) Value

Get the current value associated with a key or nil if the key is not present. The last access time of the element is updated.

func (*Cache) Put 

func (c *Cache) Put(k Key, v Value) Value

Put writes the given key and value to the map replacing any existing value if it exists. The previous value associated with the key returned or nil if the key was not present.

func (*Cache) PutIfAbsent 

func (c *Cache) PutIfAbsent(k Key, v Value) Value

PutIfAbsent writes the given key and value to the cache only if the key is absent from the cache. Nil is returned if the key-value pair were written, otherwise the old value is returned.

func (*Cache) PutIfAbsentWithTimeout 

func (c *Cache) PutIfAbsentWithTimeout(k Key, v Value, timeout time.Duration) Value

PutIfAbsentWithTimeout writes the given key and value to the cache only if the key is absent from the cache. Nil is returned if the key-value pair were written, otherwise the old value is returned. The cache expiration time will be overwritten by timeout of the key being inserted.

func (*Cache) PutWithTimeout 

func (c *Cache) PutWithTimeout(k Key, v Value, timeout time.Duration) Value

PutWithTimeout writes the given key and value to the map replacing any existing value if it exists. The previous value associated with the key returned or nil if the key was not present. The cache expiration time will be overwritten by timeout of the key being inserted.

func (*Cache) Replace 

func (c *Cache) Replace(k Key, v Value) Value

Replace overwrites the value for a key only if the key exists. The old value is returned if the value is updated, otherwise nil is returned.

func (*Cache) ReplaceWithTimeout 

func (c *Cache) ReplaceWithTimeout(k Key, v Value, timeout time.Duration) Value

ReplaceWithTimeout overwrites the value for a key only if the key exists. The old value is returned if the value is updated, otherwise nil is returned. The cache expiration time will be overwritten by timeout of the key being inserted.

func (*Cache) Size 

func (c *Cache) Size() int

Size returns the number of elements in the cache. The number includes both active elements and expired elements that have not been cleaned up.

func (*Cache) StartJanitor 

func (c *Cache) StartJanitor(interval time.Duration)

StartJanitor starts a goroutine that will periodically invoke the cache's CleanUp() method.

func (*Cache) StopJanitor 

func (c *Cache) StopJanitor()

StopJanitor stops the goroutine created by StartJanitor.

type CmdlineTuple 

type CmdlineTuple struct {
	Src, Dst []byte
}

Source and destination process names, as found by the proc module.

type Config 

type Config ucfg.Config

func LoadFile 

func LoadFile(path string) (*Config, error)

func LoadFiles 

func LoadFiles(paths ...string) (*Config, error)

func MergeConfigs 

func MergeConfigs(cfgs ...*Config) (*Config, error)

func NewConfig 

func NewConfig() *Config

func NewConfigFrom 

func NewConfigFrom(from interface{}) (*Config, error)

func NewConfigWithYAML 

func NewConfigWithYAML(in []byte, source string) (*Config, error)

func NewFlagConfig 

func NewFlagConfig(
	set *flag.FlagSet,
	def *Config,
	name string,
	usage string,
) *Config

func (*Config) Bool 

func (c *Config) Bool(name string, idx int) (bool, error)

func (*Config) Child 

func (c *Config) Child(name string, idx int) (*Config, error)

func (*Config) CountField 

func (c *Config) CountField(name string) (int, error)

func (*Config) Enabled 

func (c *Config) Enabled() bool

func (*Config) Float 

func (c *Config) Float(name string, idx int) (float64, error)

func (*Config) GetFields 

func (c *Config) GetFields() []string

func (*Config) HasField 

func (c *Config) HasField(name string) bool

func (*Config) Int 

func (c *Config) Int(name string, idx int) (int64, error)

func (*Config) Merge 

func (c *Config) Merge(from interface{}) error

func (*Config) Path 

func (c *Config) Path() string

func (*Config) PathOf 

func (c *Config) PathOf(field string) string

func (*Config) SetBool 

func (c *Config) SetBool(name string, idx int, value bool) error

func (*Config) SetChild 

func (c *Config) SetChild(name string, idx int, value *Config) error

func (*Config) SetFloat 

func (c *Config) SetFloat(name string, idx int, value float64) error

func (*Config) SetInt 

func (c *Config) SetInt(name string, idx int, value int64) error

func (*Config) SetString 

func (c *Config) SetString(name string, idx int, value string) error

func (*Config) String 

func (c *Config) String(name string, idx int) (string, error)

func (*Config) Unpack 

func (c *Config) Unpack(to interface{}) error

type Endpoint 

type Endpoint struct {
	Ip      string
	Port    uint16
	Name    string
	Cmdline string
	Proc    string
}

Endpoint represents an endpoint in the communication.

type EventMetadata 

type EventMetadata struct {
	Fields          MapStr
	FieldsUnderRoot bool `config:"fields_under_root"`
	Tags            []string
}

EventMetadata contains fields and tags that can be added to an event via configuration.

type Eventer 

type Eventer interface {
	// Add fields to MapStr.
	Event(event MapStr) error
}

Eventer defines a type its ability to fill a MapStr.

type Geoip 

type Geoip struct {
	Paths *[]string
}

Geoip represents a string slice of GeoIP paths

type HashableIpPortTuple 

type HashableIpPortTuple [MaxIpPortTupleRawSize]byte

type HashableTcpTuple 

type HashableTcpTuple [MaxTcpTupleRawSize]byte

type IpPortTuple 

type IpPortTuple struct {
	Ip_length          int
	Src_ip, Dst_ip     net.IP
	Src_port, Dst_port uint16
	// contains filtered or unexported fields
}

func NewIpPortTuple 

func NewIpPortTuple(ip_length int, src_ip net.IP, src_port uint16,
	dst_ip net.IP, dst_port uint16) IpPortTuple

func (*IpPortTuple) ComputeHashebles 

func (t *IpPortTuple) ComputeHashebles()

func (*IpPortTuple) Hashable 

func (t *IpPortTuple) Hashable() HashableIpPortTuple

Hashable returns a hashable value that uniquely identifies the IP-port tuple.

func (*IpPortTuple) RevHashable 

func (t *IpPortTuple) RevHashable() HashableIpPortTuple

Hashable returns a hashable value that uniquely identifies the IP-port tuple after swapping the source and destination.

func (*IpPortTuple) String 

func (t *IpPortTuple) String() string

type Key 

type Key interface{}

Key type used in the cache.

type MapStr 

type MapStr map[string]interface{}

Commonly used map of things, used in JSON creation and the like.

func ConvertToGenericEvent 

func ConvertToGenericEvent(v MapStr) MapStr

func MapStrUnion 

func MapStrUnion(dict1 MapStr, dict2 MapStr) MapStr

MapStrUnion creates a new MapStr containing the union of the key-value pairs of the two maps. If the same key is present in both, the key-value pairs from dict2 overwrite the ones from dict1.

func MarshallUnmarshall 

func MarshallUnmarshall(v interface{}) (MapStr, error)

func (MapStr) Clone 

func (m MapStr) Clone() MapStr

func (MapStr) CopyFieldsTo 

func (m MapStr) CopyFieldsTo(to MapStr, key string) error

func (MapStr) Delete 

func (m MapStr) Delete(key string) error

func (MapStr) EnsureCountField 

func (m MapStr) EnsureCountField() error

EnsureCountField sets the 'count' field to 1 if count does not already exist.

func (MapStr) EnsureTimestampField 

func (m MapStr) EnsureTimestampField(now func() time.Time) error

Checks if a timestamp field exists and if it doesn't it adds one by using the injected now() function as a time source.

func (MapStr) GetValue 

func (m MapStr) GetValue(key string) (interface{}, error)

func (MapStr) HasKey 

func (m MapStr) HasKey(key string) (bool, error)

func (MapStr) String 

func (m MapStr) String() string

String returns the MapStr as a JSON string.

func (MapStr) StringToPrint 

func (m MapStr) StringToPrint() string

func (MapStr) Update 

func (m MapStr) Update(d MapStr)

Update copies all the key-value pairs from the d map overwriting any existing keys.

type NetString 

type NetString []byte

NetString store the byte length of the data that follows, making it easier to unambiguously pass text and byte data between programs that could be sensitive to values that could be interpreted as delimiters or terminators (such as a null character).

func (NetString) MarshalText 

func (n NetString) MarshalText() ([]byte, error)

MarshalText exists to implement encoding.TextMarshaller interface to treat []byte as raw string by other encoders/serializers (e.g. JSON)

type RemovalListener 

type RemovalListener func(k Key, v Value)

RemovalListener is the callback function type that can be registered with the cache to receive notification of the removal of expired elements.

type TcpTuple 

type TcpTuple struct {
	Ip_length          int
	Src_ip, Dst_ip     net.IP
	Src_port, Dst_port uint16
	Stream_id          uint32
	// contains filtered or unexported fields
}

func TcpTupleFromIpPort 

func TcpTupleFromIpPort(t *IpPortTuple, tcp_id uint32) TcpTuple

func (*TcpTuple) ComputeHashebles 

func (t *TcpTuple) ComputeHashebles()

func (*TcpTuple) Hashable 

func (t *TcpTuple) Hashable() HashableTcpTuple

Hashable() returns a hashable value that uniquely identifies the TCP tuple.

func (TcpTuple) IpPort 

func (t TcpTuple) IpPort() *IpPortTuple

Returns a pointer to the equivalent IpPortTuple.

func (TcpTuple) String 

func (t TcpTuple) String() string

type Time 

type Time time.Time

Time is an abstraction for the time.Time type

func MustParseTime 

func MustParseTime(timespec string) Time

MustParseTime is a convenience equivalent of the ParseTime function that panics in case of errors.

func ParseTime 

func ParseTime(timespec string) (Time, error)

ParseTime parses a time in the TsLayout format.

func (Time) MarshalJSON 

func (t Time) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler interface. The time is a quoted string in the JsTsLayout format.

func (*Time) UnmarshalJSON 

func (t *Time) UnmarshalJSON(data []byte) (err error)

UnmarshalJSON implements js.Unmarshaler interface. The time is expected to be a quoted string in TsLayout format.

type Value 

type Value interface{}

Value type held in the cache. Cannot be nil.

Source Files

View all Source files

Directories

Path Synopsis
dtfmt package provides time formatter support with pattern syntax mostly similar to joda DateTimeFormat.
 dtfmt package provides time formatter support with pattern syntax mostly similar to joda DateTimeFormat.
The streambuf module provides helpers for buffering multiple packet payloads and some general parsing functions.
 The streambuf module provides helpers for buffering multiple packet payloads and some general parsing functions.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.