ipfix

package module
v1.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 13, 2019 License: MIT Imports: 14 Imported by: 0

README

ipfix

Package ipfix implements an IPFIX (RFC 5101) parser and interpreter.

API Documentation MIT License

Input data from an io.Reader or a []byte is parsed and chunked into messages. Template management and the standard IPFIX types are implemented so a fully parsed data set can be produced. Vendor fields can be added at runtime.

Example

To read an IPFIX stream, create a Session and then use ParseBuffer to parse data coming from a single UDP packet or similar.

var conn net.PacketConn // from somewhere
buf := make([]byte, 65507) // maximum UDP payload length
s := ipfix.NewSession()
for {
    n, _, err := conn.ReadFrom(buf)
    // handle err
    msg, err := s.ParseBuffer(buf[:n])
    // handle msg and err
}

To interpret records for correct data types and field names, use an interpreter:

i := ipfix.NewInterpreter(s)
var fieldList []ipfix.InterpretedField
for _, rec := range msg.DataRecords {
    fieldList = i.InterpretInto(rec, fieldList[:cap(fieldList)])
    // handle the field list
}

To add a vendor field to the dictionary so that it will be resolved by Interpret, create a DictionaryEntry and call AddDictionaryEntry.

e := ipfix.DictionaryEntry{
    Name: "someVendorField",
    FieldId: 42,
    EnterpriseId: 123456,
    Type: ipfix.Int32
}
i.AddDictionaryEntry(e)

License

The MIT license.

Usage

See the documentation.

Documentation

Overview

Package ipfix implements an IPFIX (RFC 5101) parser and interpreter.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	ErrNilCallback = errors.New("nil callback")
)
View Source 
var ErrProtocol = errors.New("protocol error")

ErrProtocol is returned when impossible values that constitute a protocol error are encountered.

View Source 
var ErrRead = errors.New("short read - malformed packet?")

ErrRead is returned when a packet is not long enough for the field it is supposed to contain. This is a sign of an earlier read error or a corrupted packet.

View Source 
var ErrUnknownTemplate = errors.New("unknown template")

ErrUnknownTemplate is returned when a message's data set refers to a template which has not yet been seen by the parser.

View Source 
var ErrVersion = errors.New("incorrect version field in message header - out of sync?")

The version field in IPFIX messages should always have the value 10. If it does not, you get this error. It's probably a sign of a bug in the parser or the exporter and that we have lost synchronization with the data stream. Reestablishing the session is the only way forward at this point.

View Source 
var FieldTypes = map[string]FieldType{
	"unsigned8":            Uint8,
	"unsigned16":           Uint16,
	"unsigned24":           Uint24,
	"unsigned32":           Uint32,
	"unsigned64":           Uint64,
	"signed8":              Int8,
	"signed16":             Int16,
	"signed32":             Int32,
	"signed64":             Int64,
	"float32":              Float32,
	"float64":              Float64,
	"boolean":              Boolean,
	"macAddress":           MacAddress,
	"octetArray":           OctetArray,
	"string":               String,
	"dateTimeSeconds":      DateTimeSeconds,
	"dateTimeMilliseconds": DateTimeMilliseconds,
	"dateTimeMicroseconds": DateTimeMicroseconds,
	"dateTimeNanoseconds":  DateTimeNanoseconds,
	"ipv4Address":          Ipv4Address,
	"ipv6Address":          Ipv6Address,
	"varint":               VarInt,
}

FieldTypes maps string representations of field types into their corresponding FieldType value.

Functions

func IpfixIDLookup added in v1.4.1 

func IpfixIDLookup(enterpriseID uint32, fieldID uint16) (string, bool)

IpfixIDLookup looks up the name corresponding to the specified enterprise & field IDs. It will return false if the name is not found.

func IpfixNameLookup added in v1.4.1 

func IpfixNameLookup(name string) (uint32, uint16, bool)

IpfixNameLookup looks in the built-in IPFIX dictionary for an entry matching the given name, returning the enterprise ID, field ID, and 'true' if found / 'false' if not found. e.g. calling with "sourceIPv4Address" would return 0, 8, true.

func LookupAndIdentify added in v1.4.1 

func LookupAndIdentify(name string) (uint32, uint16, uint16, bool)

LookupAndIdentify looks up the given name, returns an enterprise ID & field ID and a version number for the protocol to which it belongs: 0x09 for Netflow V9, 0x0a for IPFIX. The fourth return value is an "ok" value, indicating if either lookup was successful or not.

func NetflowV9IDLookup added in v1.4.1 

func NetflowV9IDLookup(fieldID uint16) (string, bool)

NetflowV9IDLookup looks up the name corresponding to the specified field ID. It will return false if the name is not found.

func NetflowV9NameLookup added in v1.4.1 

func NetflowV9NameLookup(name string) (uint16, bool)

NetflowV9NameLookup looks in the built-in Netflow v9 dictionary for an entry matching the given name, returning the corresponding field ID and 'true' if found / 'false' if not found. e.g. calling with "FLOWS" would return 3, true. See https://tools.ietf.org/html/rfc3954#section-8 for a list.

Types

type DataRecord 

type DataRecord struct {
	TemplateID uint16
	Fields     [][]byte
}

The DataRecord represents a single exported flow. The Fields each describe different aspects of the flow (source and destination address, counters, service, etc.).

type DictionaryEntry 

type DictionaryEntry struct {
	Name         string
	FieldID      uint16
	EnterpriseID uint32
	Type         FieldType
}

DictionaryEntry provides a mapping between an (Enterprise, Field) pair and a Name and Type.

type FieldType 

type FieldType int

FieldType is the IPFIX type of an Information Element ("Field"). 

const (
	Unknown FieldType = iota
	Uint8
	Uint16
	Uint32
	Uint24
	Uint64
	Int8
	Int16
	Int32
	Int64
	Float32
	Float64
	Boolean
	MacAddress
	OctetArray
	String
	DateTimeSeconds
	DateTimeMilliseconds
	DateTimeMicroseconds
	DateTimeNanoseconds
	Ipv4Address
	Ipv6Address
	VarInt
)

The available field types as defined by RFC 5102.

func IPfixIDTypeLookup added in v1.4.1 

func IPfixIDTypeLookup(enterpriseID uint32, fieldID uint16) (ft FieldType, ok bool)

IPfixIDTypeLookup looks up the type corresponding to the specified enterprised ID and field

func NetflowV9IDTypeLookup added in v1.4.1 

func NetflowV9IDTypeLookup(fieldID uint16) (ft FieldType, ok bool)

NetflowV9IDTypeLookup looks up the type corresponding to the specified enterprised ID and field

func (*FieldType) UnmarshalText 

func (f *FieldType) UnmarshalText(bs []byte) error

type Filter added in v1.4.1 

type Filter struct {
	HeaderFilter
	// contains filtered or unexported fields
}

func (*Filter) Clear added in v1.4.1 

func (f *Filter) Clear(eid uint32, id uint16)

func (*Filter) ClearDomainID added in v1.4.1 

func (f *Filter) ClearDomainID()

func (*Filter) ClearVersion added in v1.4.1 

func (f *Filter) ClearVersion()

func (*Filter) FilterHeader added in v1.4.1 

func (f *Filter) FilterHeader(did uint32, ver uint16) bool

func (*Filter) IsSet added in v1.4.1 

func (f *Filter) IsSet(eid uint32, id uint16) bool

func (*Filter) Set added in v1.4.1 

func (f *Filter) Set(eid uint32, id uint16)

func (*Filter) SetDomainID added in v1.4.1 

func (f *Filter) SetDomainID(v uint32)

func (*Filter) SetVersion added in v1.4.1 

func (f *Filter) SetVersion(v uint16)

type HeaderFilter added in v1.4.1 

type HeaderFilter struct {
	Version uint16 //v9 or v10

	DomainID uint32
	// contains filtered or unexported fields
}

type InterpretedField 

type InterpretedField struct {
	Name         string
	EnterpriseID uint32
	FieldID      uint16
	Value        interface{}
	RawValue     []byte
}

An InterpretedField is a field with the field name filled in and the value converted to the appropriate type. If this is not possible (because the name and type of the field is unknown at the time of interpretation), Name will be the empty string, Value will be a nil interface and RawValue will contain the original bytes.

type InterpretedTemplateFieldSpecifier 

type InterpretedTemplateFieldSpecifier struct {
	Name string
	TemplateFieldSpecifier
}

An InterpretedTemplateFieldSpecifier is a template specifier with the field name filled in, if found in the dictionary.

type Interpreter 

type Interpreter struct {
	// contains filtered or unexported fields
}

Interpreter provides translation between the raw bytes of a DataRecord and the actual values as specified by the corresponding template.

Example 
package main

import (
	"fmt"
	"os"

	"github.com/floren/ipfix"
)

func main() {
	s := ipfix.NewSession()
	i := ipfix.NewInterpreter(s)

	for {
		// ParseReader will block until a full message is available.
		msg, err := s.ParseReader(os.Stdin)
		if err != nil {
			panic(err)
		}

		var fieldList []ipfix.InterpretedField
		for _, record := range msg.DataRecords {
			fieldList = i.InterpretInto(record, fieldList)
			fmt.Println(fieldList)
		}
	}
}

Output:

func NewInterpreter 

func NewInterpreter(s *Session) *Interpreter

NewInterpreter craets a new Interpreter based on the specified Session. It will attempt to use the appropriate dictionary (IPFIX or NFv9) based on the Session's most-recently parsed message.

func NewInterpreterVersion added in v1.4.1 

func NewInterpreterVersion(s *Session, v uint16) (*Interpreter, error)

func (*Interpreter) AddDictionaryEntry 

func (i *Interpreter) AddDictionaryEntry(e DictionaryEntry)

AddDictionaryEntry adds a DictionaryEntry (containing a vendor field) to the dictionary used by Interpret.

Example 
package main

import (
	"github.com/floren/ipfix"
)

func main() {
	s := ipfix.NewSession()
	i := ipfix.NewInterpreter(s)

	entry := ipfix.DictionaryEntry{
		Name:         "someVendorField",
		FieldID:      42,
		EnterpriseID: 123456,
		Type:         ipfix.Int32,
	}

	i.AddDictionaryEntry(entry)

	// Now use i.Interpret() etc as usual.
}

Output:

func (*Interpreter) Interpret 

func (i *Interpreter) Interpret(rec DataRecord) []InterpretedField

Interpret a raw DataRecord into a list of InterpretedFields.

func (*Interpreter) InterpretInto 

func (i *Interpreter) InterpretInto(rec DataRecord, fieldList []InterpretedField) []InterpretedField

InterpretInto interprets a raw DataRecord into an existing slice of InterpretedFields. If the slice is not long enough it will be reallocated.

func (*Interpreter) InterpretTemplate 

func (i *Interpreter) InterpretTemplate(rec TemplateRecord) []InterpretedTemplateFieldSpecifier

InterpretTemplate interprets a template record and adds a name to the interpreted fields, if a given {EnterpriseID,FieldID} can be find in the dictionary.

type Message 

type Message struct {
	Header          MessageHeader
	DataRecords     []DataRecord
	TemplateRecords []TemplateRecord
}

A Message is the top level construct representing an IPFIX message. A well formed message contains one or more sets of data or template information.

type MessageHeader 

type MessageHeader struct {
	Version        uint16 // Always 0x09 or 0x0a
	Length         uint16
	SysUptime      uint32 // NetflowV9 only
	ExportTime     uint32 // Epoch seconds
	SequenceNumber uint32
	DomainID       uint32 // "source ID" in netflow v9
}

The MessageHeader provides metadata for the entire Message. The sequence number and domain ID can be used to gain knowledge of messages lost on an unreliable transport such as UDP.

func Read 

func Read(r io.Reader, bs []byte) ([]byte, MessageHeader, error)

Read reads and returns an IPFIX message, the parsed message header and an error or nil. The given byte slice is used and returned (sliced to the message length) if it is large enough to contain the message; otherwise a new slice is allocated. The returned message slice contains the message header.

type Option added in v1.1.0 

type Option func(*Session)

An option can be passed to New()

func WithIDAliasing added in v1.1.0 

func WithIDAliasing(v bool) Option

WithIDAliasing enables or disables template id aliasing. The default is disabled.

type Record added in v1.4.1 

type Record struct {
	MessageHeader
	SetID        int
	DataRecordID int
	EndOfRecord  bool
	Err          error
}

type RecordCallback added in v1.4.1 

type RecordCallback func(*Record, uint32, uint16, []byte) error

type Session 

type Session struct {
	// contains filtered or unexported fields
}

The Session is the context for IPFIX messages.

Example 
package main

import (
	"fmt"
	"os"

	"github.com/floren/ipfix"
)

func main() {
	s := ipfix.NewSession()

	for {
		// ParseReader will block until a full message is available.
		msg, err := s.ParseReader(os.Stdin)
		if err != nil {
			panic(err)
		}

		for _, record := range msg.DataRecords {
			// record contains raw enterpriseId, fieldId => []byte information
			fmt.Println(record)
		}
	}
}

Output:

func NewSession 

func NewSession(opts ...Option) *Session

NewSession initializes a new Session based on the provided io.Reader.

func (*Session) ExportTemplateRecords added in v1.3.0 

func (s *Session) ExportTemplateRecords() []TemplateRecord

func (*Session) LoadTemplateRecords added in v1.3.0 

func (s *Session) LoadTemplateRecords(trecs []TemplateRecord)

func (*Session) LookupTemplateRecords added in v1.4.1 

func (s *Session) LookupTemplateRecords(m Message) ([]TemplateRecord, error)

LookupTemplateRecords returns the list of template records referred to by the data sets in the given message. It also includes any templates already extant in the message.

func (*Session) Marshal added in v1.4.1 

func (s *Session) Marshal(m Message) ([]byte, error)

Marshall a Message struct back into a raw IPFIX buffer

func (*Session) ParseBuffer 

func (s *Session) ParseBuffer(bs []byte) (Message, error)

ParseBuffer extracts one message (IPFIX or Netflow V9) from the given buffer and returns it. Err is nil if the buffer could be parsed correctly. ParseBuffer is goroutine safe.

func (*Session) ParseBufferAll added in v1.2.0 

func (s *Session) ParseBufferAll(bs []byte) ([]Message, error)

ParseBufferAll extracts all message from the given buffer and returns them. Err is nil if the buffer could be parsed correctly. ParseBufferAll is goroutine safe. ParseBufferAll does not currently support NFv9

func (*Session) ParseReader deprecated 

func (s *Session) ParseReader(r io.Reader) (Message, error)

ParseReader extracts and returns one message from the IPFIX stream. As long as err is nil, further messages can be read from the stream. Errors are not recoverable -- once an error has been returned, ParseReader should not be called again on the same session.

Deprecated: use ParseBuffer instead. ParseReader does not support Netflow v9

func (*Session) Version added in v1.4.1 

func (s *Session) Version() uint16

Version returns the Netflow/IPFIX version seen in the most recent header. It returns 0x09 for Netflow v9 and 0x0a for IPFIX. It defaults to IPFIX (0x0a) if no messages have been parsed yet.

type TemplateFieldSpecifier 

type TemplateFieldSpecifier struct {
	EnterpriseID uint32
	FieldID      uint16
	Length       uint16
}

The TemplateFieldSpecifier describes the ID and size of the corresponding Fields in a DataRecord.

type TemplateRecord 

type TemplateRecord struct {
	TemplateID      uint16
	FieldSpecifiers []TemplateFieldSpecifier
}

The TemplateRecord describes a data template, as used by DataRecords.

type Walker added in v1.4.1 

type Walker struct {
	// contains filtered or unexported fields
}

func NewWalker added in v1.4.1 

func NewWalker(f *Filter, trbufsize, fidbufsize int) (w *Walker, err error)

NewWalker creates a new Walker object. It will use the given Filter to perform pre-filtering of entries. trbufsize and fidbufsize give sizes to use when pre-allocating the template record buffer and the template field specifier buffer, respectively; 64 and 4096 would be safe defaults to use.

func (*Walker) SetHeaderOnly added in v1.4.1 

func (w *Walker) SetHeaderOnly(v bool)

SetHeaderOnly can be used to enable header-only parsing via the walker. If set to true, the WalkBuffer function will call the callback only once, with an EID and FID of 0.

func (*Walker) WalkBuffer added in v1.4.1 

func (w *Walker) WalkBuffer(buf []byte, cb RecordCallback) (err error)

WalkBuffer walks an IPFIX or Netflow V9 packet in buf, calling the callback function in accordance with the following rules:

1. If SetHeaderOnly(true) was called, the callback will be called precisely once, with EndOfRecord set in the record parameter, EID and FID set to zero, and a nil buffer. This allows code to view the packet header only.

2. If a nil Filter was passed when building the Walker, the callback will be called for every field in each record. When a record has been fully processed, the callback will be called again with EndOfRecord set to true, EID and FID set to zero, and a nil buffer. The next record will then be processed, and so on until the message has been fully read.

3. If a non-nil Filter was passed, the function will behave exactly as in case #2 except that only those EID and FID combinations registered with the Filter will trigger a callback. The EndOfRecord callback will still occur as normal.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.