parsetypes

package
v3.2.1+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 25, 2020 License: GPL-3.0 Imports: 4 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// BOOL reflects true or false, designated 'T' or 'F'
	Bool = "bool"

	// COUNT is a numeric representation of a UINT_64 represented as either
	// a string of digits or a hex number. Note that hex numbers will begin
	// with the traditional 0x
	Count = "count"

	// INT is a numeric type representing an INT_64 represetned by a string
	// of digits preceded by either a '+' or a '-'. Note that INT may also
	// be expressed in hex and will maintain its leading sign ('-0xff')
	Int = "int"

	// DOUBLE is a numeric type representing a double-precision float.
	// Representation is a string of digits with an optional decimal point
	// as well as optional '+' or '-' proceeding the number. The number may
	// also be optionally scaled with e notation. So 1234 123.4 -123.4
	// +1.234 and .003E-23 are examples of valid double types.
	Double = "double"

	// TIME is a temporal type representing an absolute time. Until found
	// otherwise it will be assumed that all time values are UNIX-NANO.
	Time = "time"

	// INTERVAL is a temporal type representing relative time. An Interval
	// constant is represented by by a numeric constant followed by a time
	// unit which is one of usec, msec, sec, min, hr, or day. An 's' may
	// be appended to the unit so 3.5 min and 3.5mins represent the same
	// value. Finally an optional '-' negates an interval, denoting past
	// time. So -12 hr is read as "twelve hours in the past."
	Interval = "interval"

	// STRING is a type used to hold character string values.
	String = "string"

	// PATTERN is a type used to represent regular expressions. Pattern
	// documentation can be found at
	// http://flex.sourceforge.net/manual/Patterns.html
	Pattern = "pattern"

	// PORT is a type used to represent transport-level port numbers these
	// are typically represented as a number followed by one of /udp, /tcp,
	// /icmp, or /unkown.
	Port = "port"

	// ADDR is a type used to represent an IP address. IPv4 addresses are
	// represented in dotted quad notation. IPv6 addresses are written in
	// colon hex notation as outlined in RFC 2373 (including the mixed
	// notation which allows dotted quad IPv4 addresses in the lower 32
	// bits) and further placed into brackets. So [::ffff:192.168.1.100]
	// can be used to represent the IPv4 address 192.168.1.100.
	Addr = "addr"

	// SUBNET is a type used to represent a subnet in CIDR notation. So
	// 10.10.150.0/24 and [fe80::]/64 are valid subnets.
	Subnet = "subnet"

	// ENUM is a type allowing the specification of a set of related
	// values that have no further structure.
	Enum = "enum"

	// STRING_SET is a SET which contains STRINGs
	StringSet = "set[string]"

	// ENUM_SET is a SET which contains ENUMs
	EnumSet = "set[enum]"

	// STRING_VECTOR is a VECTOR which contains STRINGs
	StringVector = "vector[string]"

	// INTERVAL_VECTOR is a VECTOR which contains INTERVALs
	IntervalVector = "vector[interval]"

	// FUNCTION represents a function type in bro script.
	Function = "function"

	// EVENT represents an event handler in bro script.
	Event = "event"

	// HOOK represents a bro script object best described as as the an
	// intersection of a function and an event.
	Hook = "hook"

	// A file object which can be written to, but not read from (which is a
	// limitation of bro script and has nothing to do with brosync).
	File = "file"

	// OPAQUE represents data whos type is intentionally hidden, but whose
	// values may be passed to certain bro script builtins.
	Opaque = "opaque"

	// ANY is used to bypass strong typing in bro script.
	Any = "any"
)

Further documentation on bros datatypes can be found on the bro website at: https://www.bro.org/sphinx/script-reference/types.html It is of value to note that many of these types have applications specific to bro script and will likely never be implemented as types with any meaning in ai-hunt.

Variables

This section is empty.

Functions

func NewBroDataFactory 

func NewBroDataFactory(fileType string) func() BroData

NewBroDataFactory creates a new BroData based on the string which appears in that log's objType field

Types

type BroData 

type BroData interface {
	TargetCollection(*config.StructureTableCfg) string
	Indices() []string
	// ConvertFromJSON should be called after importing from JSON logs
	ConvertFromJSON()
}

BroData holds a line of a bro log

type Conn 

type Conn struct {
	// ID is the id coming out of mongodb
	ID bson.ObjectId `bson:"_id,omitempty"`
	// TimeStamp of this connection
	TimeStamp int64 `bson:"ts" bro:"ts" brotype:"time" json:"-"`
	// TimeStampGeneric is used when reading from json files
	TimeStampGeneric interface{} `bson:"-" json:"ts"`
	// UID is the Unique Id for this connection (generated by Bro)
	UID string `bson:"uid" bro:"uid" brotype:"string" json:"uid"`
	// Source is the source address for this connection
	Source string `bson:"id_orig_h" bro:"id.orig_h" brotype:"addr" json:"id.orig_h"`
	// SourcePort is the source port of this connection
	SourcePort int `bson:"id_orig_p" bro:"id.orig_p" brotype:"port" json:"id.orig_p"`
	// Destination is the destination of the connection
	Destination string `bson:"id_resp_h" bro:"id.resp_h" brotype:"addr" json:"id.resp_h"`
	// DestinationPort is the port at the destination host
	DestinationPort int `bson:"id_resp_p" bro:"id.resp_p" brotype:"port" json:"id.resp_p"`
	// Proto is the string protocol identifier for this connection
	Proto string `bson:"proto" bro:"proto" brotype:"enum" json:"proto"`
	// Service describes the service of this connection if there was one
	Service string `bson:"service" bro:"service" brotype:"string" json:"service"`
	// Duration is the floating point representation of connection length
	Duration float64 `bson:"duration" bro:"duration" brotype:"interval" json:"duration"`
	// OrigBytes is the byte count coming from the origin
	OrigBytes int64 `bson:"orig_bytes" bro:"orig_bytes" brotype:"count" json:"orig_bytes"`
	// RespBytes is the byte count coming in on response
	RespBytes int64 `bson:"resp_bytes" bro:"resp_bytes" brotype:"count" json:"resp_bytes"`
	// ConnState has data describing the state of a connection
	ConnState string `bson:"conn_state" bro:"conn_state" brotype:"string" json:"conn_state"`
	// LocalOrigin denotes that the connection originated locally
	LocalOrigin bool `bson:"local_orig" bro:"local_orig" brotype:"bool" json:"local_orig"`
	// LocalResponse denote that the connection responded locally
	LocalResponse bool `bson:"local_resp" bro:"local_resp" brotype:"bool" json:"local_resp"`
	// MissedBytes keeps a count of bytes missed
	MissedBytes int64 `bson:"missed_bytes" bro:"missed_bytes" brotype:"count" json:"missed_bytes"`
	// History is a string containing historical information
	History string `bson:"history"  bro:"history" brotype:"string" json:"history"`
	// OrigPkts is a count of origin packets
	OrigPkts int64 `bson:"orig_pkts"  bro:"orig_pkts" brotype:"count" json:"orig_pkts"`
	// OrigIpBytes is another origin data count
	OrigIPBytes int64 `bson:"orig_ip_bytes" bro:"orig_ip_bytes" brotype:"count" json:"orig_ip_bytes"`
	// RespPkts counts response packets
	RespPkts int64 `bson:"resp_pkts" bro:"resp_pkts" brotype:"count" json:"resp_pkts"`
	// RespIpBytes gives the bytecount of response data
	RespIPBytes int64 `bson:"resp_ip_bytes" bro:"resp_ip_bytes" brotype:"count" json:"resp_ip_bytes"`
	// TunnelParents lists tunnel parents
	TunnelParents []string `bson:"tunnel_parents" bro:"tunnel_parents" brotype:"set[string]" json:"tunnel_parents"`
}

Conn provides a data structure for bro's connection data

func (*Conn) ConvertFromJSON 

func (line *Conn) ConvertFromJSON()

ConvertFromJSON performs any extra conversions necessary when reading from JSON

func (*Conn) Indices 

func (line *Conn) Indices() []string

Indices gives MongoDB indices that should be used with the collection

func (*Conn) TargetCollection 

func (line *Conn) TargetCollection(config *config.StructureTableCfg) string

TargetCollection returns the mongo collection this entry should be inserted

type DNS 

type DNS struct {
	// ID contains the id set by mongodb
	ID bson.ObjectId `bson:"_id,omitempty"`
	// TimeStamp of this connection
	TimeStamp int64 `bson:"ts" bro:"ts" brotype:"time" json:"-"`
	// TimeStampGeneric is used when reading from json files
	TimeStampGeneric interface{} `bson:"-" json:"ts"`
	// UID is the Unique Id for this connection (generated by Bro)
	UID string `bson:"uid" bro:"uid" brotype:"string" json:"uid"`
	// Source is the source address for this connection
	Source string `bson:"id_orig_h" bro:"id.orig_h" brotype:"addr" json:"id.orig_h"`
	// SourcePort is the source port of this connection
	SourcePort int `bson:"id_orig_p" bro:"id.orig_p" brotype:"port" json:"id.orig_p"`
	// Destination is the destination of the connection
	Destination string `bson:"id_resp_h" bro:"id.resp_h" brotype:"addr" json:"id.resp_h"`
	// DestinationPort is the port at the destination host
	DestinationPort int `bson:"id_resp_p" bro:"id.resp_p" brotype:"port" json:"id.resp_p"`
	// Proto is the string protocol identifier for this connection
	Proto string `bson:"proto" bro:"proto" brotype:"enum" json:"proto"`
	// TransID contains a 16 bit identifier assigned by the program that generated
	// the query
	TransID int64 `bson:"trans_id" bro:"trans_id" brotype:"count" json:"trans_id"`
	// RTT contains the round trip time of this request / response
	RTT float64 `bson:"rtt" bro:"rtt" brotype:"interval" json:"rtt"`
	// Query contains the query string
	Query string `bson:"query" bro:"query" brotype:"string" json:"query"`
	// QClass contains a the qclass of the query
	QClass int64 `bson:"qclass" bro:"qclass" brotype:"count" json:"qclass"`
	// QClassName contains a descriptive name for the query
	QClassName string `bson:"qclass_name" bro:"qclass_name" brotype:"string" json:"qclass_name"`
	// QType contains the value of the query type
	QType int64 `bson:"qtype" bro:"qtype" brotype:"count" json:"qtype"`
	// QTypeName provides a descriptive name for the query
	QTypeName string `bson:"qtype_name" bro:"qtype_name" brotype:"string" json:"qtype_name"`
	// RCode contains the response code value from the DNS messages
	RCode int64 `bson:"rcode" bro:"rcode" brotype:"count" json:"rcode"`
	// RCodeName provides a descriptive name for RCode
	RCodeName string `bson:"rcode_name" bro:"rcode_name" brotype:"string" json:"rcode_name"`
	// AA represents the state of the authoritive answer bit of the resp messages
	AA bool `bson:"AA" bro:"AA" brotype:"bool" json:"AA"`
	// TC represents the truncation bit of the message
	TC bool `bson:"TC" bro:"TC" brotype:"bool" json:"TC"`
	// RD represens the recursion desired bit of the message
	RD bool `bson:"RD" bro:"RD" brotype:"bool" json:"RD"`
	// RA represents the recursion available bit of the message
	RA bool `bson:"RA" bro:"RA" brotype:"bool" json:"RA"`
	// Z represents the state of a reseverd field that should be zero in qll queries
	Z int64 `bson:"Z" bro:"Z" brotype:"count" json:"Z"`
	// Answers contains the set of resource descriptions in the query answer
	Answers []string `bson:"answers" bro:"answers" brotype:"vector[string]" json:"answers"`
	// TTLs contains a vector of interval type time to live values
	TTLs []float64 `bson:"TTLs" bro:"TTLs" brotype:"vector[interval]" json:"TTLs"`
	// Rejected indicates if this query was rejected or not
	Rejected bool `bson:"rejected" bro:"rejected" brotype:"bool" json:"rejected"`
}

DNS provides a data structure for entries in the bro DNS log

func (*DNS) ConvertFromJSON 

func (line *DNS) ConvertFromJSON()

ConvertFromJSON performs any extra conversions necessary when reading from JSON

func (*DNS) Indices 

func (line *DNS) Indices() []string

Indices gives MongoDB indices that should be used with the collection

func (*DNS) TargetCollection 

func (line *DNS) TargetCollection(config *config.StructureTableCfg) string

TargetCollection returns the mongo collection this entry should be inserted

type HTTP 

type HTTP struct {
	// ID is the object id as set by mongodb
	ID bson.ObjectId `bson:"_id,omitempty"`
	// TimeStamp of this connection
	TimeStamp int64 `bson:"ts" bro:"ts" brotype:"time" json:"-"`
	// TimeStampGeneric is used when reading from json files
	TimeStampGeneric interface{} `bson:"-" json:"ts"`
	// UID is the Unique Id for this connection (generated by Bro)
	UID string `bson:"uid" bro:"uid" brotype:"string" json:"uid"`
	// Source is the source address for this connection
	Source string `bson:"id_orig_h" bro:"id.orig_h" brotype:"addr" json:"id.orig_h"`
	// SourcePort is the source port of this connection
	SourcePort int `bson:"id_orig_p" bro:"id.orig_p" brotype:"port" json:"id.orig_p"`
	// Destination is the destination of the connection
	Destination string `bson:"id_resp_h" bro:"id.resp_h" brotype:"addr" json:"id.resp_h"`
	// DestinationPort is the port at the destination host
	DestinationPort int `bson:"id_resp_p" bro:"id.resp_p" brotype:"port" json:"id.resp_p"`
	// Transdepth is the ordinal value of requests into a pipeline transaction
	TransDepth int64 `bson:"trans_depth" bro:"trans_depth" brotype:"count" json:"trans_depth"`
	// Version is the value of version in the request
	Version string `bson:"version" bro:"version" brotype:"string" json:"version"`
	// Method is the request method used
	Method string `bson:"method" bro:"method" brotype:"string" json:"method"`
	// Host is the value of the HOST header
	Host string `bson:"host" bro:"host" brotype:"string" json:"host"`
	// URI is the uri used in this request
	URI string `bson:"uri" bro:"uri" brotype:"string" json:"uri"`
	// Referrer is the value of the referrer header in the request
	Referrer string `bson:"referrer" bro:"referrer" brotype:"string" json:"referrer"`
	// UserAgent gives the user agent from the request
	UserAgent string `bson:"user_agent" bro:"user_agent" brotype:"string" json:"user_agent"`
	// ReqLen holds the length of the request body uncompressed
	ReqLen int64 `bson:"request_body_len" bro:"request_body_len" brotype:"count" json:"request_body_len"`
	// RespLen hodls the length of the response body uncompressed
	RespLen int64 `bson:"response_body_len" bro:"response_body_len" brotype:"count" json:"response_body_len"`
	// StatusCode holds the status result
	StatusCode int64 `bson:"status_code" bro:"status_code" brotype:"count" json:"status_code"`
	// StatusMsg contains a string status message returned by the server
	StatusMsg string `bson:"status_msg" bro:"status_msg" brotype:"string" json:"status_msg"`
	// InfoCode holds the last seen 1xx informational reply code
	InfoCode int64 `bson:"info_code" bro:"info_code" brotype:"count" json:"info_code"`
	// InfoMsg holds the last seen 1xx message string
	InfoMsg string `bson:"info_msg" bro:"info_msg" brotype:"string" json:"info_msg"`
	// Tags contains a set of indicators of various attributes related to a particular req and
	// response pair
	Tags []string `bson:"tags" bro:"tags" brotype:"set[enum]" json:"tags"`
	// UserName will contain a username in the case of basic auth implementation
	UserName string `bson:"username" bro:"username" brotype:"string" json:"username"`
	// Password will contain a password in the case of basic auth implementation
	Password string `bson:"password" bro:"password" brotype:"string" json:"password"`
	// Proxied contains all headers that indicate a request was proxied
	Proxied []string `bson:"proxied" bro:"proxied" brotype:"set[string]" json:"proxied"`
	// OrigFuids contains an ordered vector of uniq file IDs
	OrigFuids []string `bson:"orig_fuids" bro:"orig_fuids" brotype:"vector[string]" json:"orig_fuids"`
	// OrigFilenames contains an ordered vector of filenames from the client
	OrigFilenames []string `bson:"orig_filenames" bro:"orig_filenames" brotype:"vector[string]" json:"orig_filenames"`
	// OrigMimeTypes contains an ordered vector of mimetypes
	OrigMimeTypes []string `bson:"orig_mime_types" bro:"orig_mime_types" brotype:"vector[string]" json:"orig_mime_types"`
	// RespFuids contains an ordered vector of unique file IDs in the response
	RespFuids []string `bson:"resp_fuids" bro:"resp_fuids" brotype:"vector[string]" json:"resp_fuids"`
	// RespFilenames contains an ordered vector of unique files in the response
	RespFilenames []string `bson:"resp_filenames" bro:"resp_filenames" brotype:"vector[string]" json:"resp_filenames"`
	// RespMimeTypes contains an ordered vector of unique MIME entities in the HTTP response body
	RespMimeTypes []string `bson:"resp_mime_types" bro:"resp_mime_types" brotype:"vector[string]" json:"resp_mime_types"`
}

HTTP provides a data structure for entries in bro's HTTP log file

func (*HTTP) ConvertFromJSON 

func (line *HTTP) ConvertFromJSON()

ConvertFromJSON performs any extra conversions necessary when reading from JSON

func (*HTTP) Indices 

func (line *HTTP) Indices() []string

Indices gives MongoDB indices that should be used with the collection

func (*HTTP) TargetCollection 

func (line *HTTP) TargetCollection(config *config.StructureTableCfg) string

TargetCollection returns the mongo collection this entry should be inserted

type SSL 

type SSL struct {
	// TimeStamp of this connection
	TimeStamp int64 `bson:"ts" bro:"ts" brotype:"time" json:"-"`
	// TimeStampGeneric is used when reading from json files
	TimeStampGeneric interface{} `bson:"-" json:"ts"`
	// UID is the Unique Id for this connection (generated by Bro)
	UID string `bson:"uid" bro:"uid" brotype:"string" json:"uid"`
	// Source is the source address for this connection
	Source string `bson:"id_orig_h" bro:"id.orig_h" brotype:"addr" json:"id.orig_h"`
	// SourcePort is the source port of this connection
	SourcePort int `bson:"id_orig_p" bro:"id.orig_p" brotype:"port" json:"id.orig_p"`
	// Destination is the destination of the connection
	Destination string `bson:"id_resp_h" bro:"id.resp_h" brotype:"addr" json:"id.resp_h"`
	// DestinationPort is the port at the destination host
	DestinationPort int `bson:"id_resp_p" bro:"id.resp_p" brotype:"port" json:"id.resp_p"`
	// VersionNum  : Numeric SSL/TLS version that the server chose
	VersionNum int `bson:"version_num" bro:"version_num" brotype:"count" json:"version_num"`
	// Version : SSL/TLS version that the server chose
	Version string `bson:"version" bro:"version" brotype:"string" json:"version"`
	// Cipher : SSL/TLS cipher suite that the server chose
	Cipher string `bson:"cipher" bro:"cipher" brotype:"string" json:"cipher"`
	// Curve : Elliptic curve the server chose when using ECDH/ECDHE
	Curve string `bson:"curve" bro:"curve" brotype:"string" json:"curve"`
	// ServerName : Value of the Server Name Indicator SSL/TLS extension.
	// It indicates the server name that the client was requesting.
	ServerName string `bson:"server_name" bro:"server_name" brotype:"string" json:"server_name"`
	// SessionID : Session ID offered by the client for session resumption.
	// Not used for logging.
	SessionID string `bson:"session_id" bro:"session_id" brotype:"string" json:"session_id"`
	// Resumed : Flag to indicate if the session was resumed reusing the key
	// material exchanged in an earlier connection
	Resumed bool `bson:"resumed" bro:"resumed" brotype:"bool" json:"resumed"`
	// ClientTicketEmptySessionSeen : Flag to indicate if we saw a non-empty
	// session ticket being sent by the client using an empty session ID.
	// This value is used to determine if a session is being resumed.
	// It’s not logged.  Note: may not be present in older bro versions.
	ClientTicketEmptySessionSeen bool `` /* 133-byte string literal not displayed */
	// ClientKeyExchangeSeen :Flag to indicate if we saw a client key exchange
	// message sent by the client. This value is used to determine if a session
	// is being resumed. It’s not logged.
	// Note: may not be present in older bro versions.
	ClientKeyExchangeSeen bool `bson:"client_key_exchange_seen" bro:"client_key_exchange_seen" brotype:"bool" json:"client_key_exchange_seen"`
	// ServerAppData : Count to track if the server already sent an application
	// data packet for TLS 1.3. Used to track when a session was established
	// Note: may not be present in older bro versions.
	ServerAppData int `bson:"server_appdata" bro:"server_appdata" brotype:"count" json:"server_appdata"`
	// ClientAppData : Flag to track if the client already sent an application
	// data packet for TLS 1.3. Used to track when a session was established
	// Note: may not be present in older bro versions.
	ClientAppData bool `bson:"client_appdata" bro:"client_appdata" brotype:"bool" json:"client_appdata"`
	// LastAlert : Last alert that was seen during the connection.
	LastAlert string `bson:"last_alert" bro:"last_alert" brotype:"string" json:"last_alert"`
	// NextProtocol : Next protocol the server chose using the application layer
	// next protocol extension, if present.
	NextProtocol string `bson:"next_protocol" bro:"next_protocol" brotype:"string" json:"next_protocol"`
	// AnalyzerID : The analyzer ID used for the analyzer instance attached to
	// each connection. It is not used for logging since it’s a meaningless
	// arbitrary number. Note: may not be present in older bro versions.
	AnalyzerID int `bson:"analyzer_id" bro:"analyzer_id" brotype:"count" json:"analyzer_id"`
	// Established : Flag to indicate if this ssl session has been established
	// successfully, or if it was aborted during the handshake
	Established bool `bson:"established" bro:"established" brotype:"bool" json:"established"`
	// Logged : Flag to indicate if this record already has been logged, to
	// prevent duplicates. Note: may not be present in older bro versions.
	Logged bool `bson:"logged" bro:"logged" brotype:"bool" json:"logged"`
	// CertChainFuids
	CertChainFuids []string `bson:"cert_chain_fuids" bro:"cert_chain_fuids" brotype:"vector[string]" json:"cert_chain_fuids"`
	// ClientCertChainFuids
	ClientCertChainFuids []string `bson:"client_cert_chain_fuids"  bro:"client_cert_chain_fuids" brotype:"vector[string]" json:"client_cert_chain_fuids"`
	// Subject
	Subject string `bson:"subject"  bro:"subject" brotype:"string" json:"subject"`
	// Issuer
	Issuer string `bson:"issuer"  bro:"issuer" brotype:"string" json:"issuer"`
	// ClientSubject
	ClientSubject string `bson:"client_subject"  bro:"client_subject" brotype:"string" json:"client_subject"`
	// ClientIssuer
	ClientIssuer string `bson:"client_issuer"  bro:"client_issuer" brotype:"string" json:"client_issuer"`
	// ValidationStatus
	ValidationStatus string `bson:"validation_status"  bro:"validation_status" brotype:"string" json:"validation_status"`
	// ValidationCode  : Numeric SSL/TLS version that the server chose
	ValidationCode int `bson:"validation_code" bro:"validation_code" brotype:"int" json:"validation_code"`
	// JA3 hash
	JA3 string `bson:"ja3" bro:"ja3" brotype:"string" json:"ja3"`
}

SSL provides a data structure for bro's connection data

func (*SSL) ConvertFromJSON 

func (line *SSL) ConvertFromJSON()

ConvertFromJSON performs any extra conversions necessary when reading from JSON

func (*SSL) Indices 

func (line *SSL) Indices() []string

Indices gives MongoDB indices that should be used with the collection

func (*SSL) TargetCollection 

func (line *SSL) TargetCollection(config *config.StructureTableCfg) string

TargetCollection returns the mongo collection this entry should be inserted

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.