collector

Documentation

Overview

Package collector provides a mechanism to collect network packets from a network interface on macOS, linux and windows

Index

• Variables
• func IsPcap(file string) (bool, error)
• func OpenPCAP(file string) (*pcapgo.Reader, *os.File, error)
• type BatchInfo
• type Collector
  • func New(config Config) *Collector
  • func (c *Collector) CloseFileHandleOnShutdown(f *os.File)
  • func (c *Collector) CollectBPF(path, bpf string) error
  • func (c *Collector) CollectLive(i string, bpf string, ctx context.Context) error
  • func (c *Collector) CollectPcap(path string) error
  • func (c *Collector) CollectPcapNG(path string) error
  • func (c *Collector) GetNumPackets() int64
  • func (c *Collector) Init() (err error)
  • func (c *Collector) InitBatching(bpf string, in string) ([]BatchInfo, *pcap.Handle, error)
  • func (c *Collector) PrintConfiguration()
  • func (c *Collector) RenderPacketsPerSecond(inputFile string, out string)
  • func (c *Collector) Stop()
• type Config

Variables

var DefaultConfig = Config{
	Workers:             1000,
	PacketBufferSize:    100,
	WriteUnknownPackets: false,
	Promisc:             false,
	SnapLen:             defaults.SnapLen,
	DPI:                 false,
	BaseLayer:           utils.GetBaseLayer("ethernet"),
	DecodeOptions:       utils.GetDecodeOptions("datagrams"),
	DecoderConfig:       config.DefaultConfig,
	ResolverConfig:      resolvers.DefaultConfig,
	Timeout:             pcap.BlockForever,
	LogErrors:           false,
}

DefaultConfig is a sane example configuration.

var DefaultConfigDPI = Config{
	Workers:             1000,
	PacketBufferSize:    100,
	WriteUnknownPackets: false,
	Promisc:             false,
	SnapLen:             defaults.SnapLen,
	DPI:                 true,
	BaseLayer:           utils.GetBaseLayer("ethernet"),
	DecodeOptions:       utils.GetDecodeOptions("datagrams"),
	DecoderConfig:       config.DefaultConfig,
	ResolverConfig:      resolvers.DefaultConfig,
	LogErrors:           false,
}

DefaultConfigDPI is a sane example configuration for use with Deep Packet Inspection.

Functions

func IsPcap

func IsPcap(file string) (bool, error)

IsPcap checks whether a file is a PCAP file.

func OpenPCAP

func OpenPCAP(file string) (*pcapgo.Reader, *os.File, error)

OpenPCAP opens a Packet Capture file.

Types

type BatchInfo

type BatchInfo struct {
	Type types.Type
	Chan <-chan []byte
}

BatchInfo contains information about a Batch source.

type Collector

type Collector struct {
	InputFile string
	PrintTime bool
	Bpf       string

	Epochs int
	// contains filtered or unexported fields
}

Collector provides an interface to collect data from PCAP or a network interface. this structure has an optimized field order to avoid excessive padding.

func New

func New(config Config) *Collector

New returns a new Collector instance.

func (*Collector) CloseFileHandleOnShutdown

func (c *Collector) CloseFileHandleOnShutdown(f *os.File)

CloseFileHandleOnShutdown allows to register file handles for close on shutdown.

func (*Collector) CollectBPF

func (c *Collector) CollectBPF(path, bpf string) error

CollectBPF open the named PCAP file and sets the specified BPF filter.

func (*Collector) CollectLive

func (c *Collector) CollectLive(i string, bpf string, ctx context.Context) error

CollectLive starts collection of data from the given interface. optionally a BPF can be supplied. this is the linux version that uses the pure go version from pcapgo to fetch packets live.

func (*Collector) CollectPcap

func (c *Collector) CollectPcap(path string) error

CollectPcap implements parallel decoding of incoming packets.

func (*Collector) CollectPcapNG

func (c *Collector) CollectPcapNG(path string) error

CollectPcapNG implements parallel decoding of incoming packets.

func (*Collector) GetNumPackets

func (c *Collector) GetNumPackets() int64

GetNumPackets returns the current number of processed packets.

func (*Collector) Init

func (c *Collector) Init() (err error)

Init sets up the collector and starts the configured number of workers must be called prior to usage of the collector instance.

func (*Collector) InitBatching

func (c *Collector) InitBatching(bpf string, in string) ([]BatchInfo, *pcap.Handle, error)

InitBatching initializes batching mode and returns an array of Batchinfos and the pcap handle closing the handle must be done by the caller.

func (*Collector) PrintConfiguration

func (c *Collector) PrintConfiguration()

PrintConfiguration dumps the current collector config to stdout.

func (*Collector) RenderPacketsPerSecond

func (c *Collector) RenderPacketsPerSecond(inputFile string, out string)

RenderPacketsPerSecond will render a html chart for the packet ingestion rate of the collector over time. Do not call while the collector is running, the access to the pps map is not synchronized. If you need runtime metrics, use prometheus.

func (*Collector) Stop

func (c *Collector) Stop()

Stop will halt packet collection and wait for all processing to finish.

type Config

type Config struct {

	// Decoder configuration
	DecoderConfig *config.Config

	// Baselayer to start decoding from
	BaseLayer gopacket.LayerType

	// Number of workers to use
	Workers int

	// Size of the input buffer channels for the workers
	PacketBufferSize int

	// Ethernet frame snaplength for live capture
	SnapLen int

	// Can be used to periodically free OS memory
	FreeOSMem int

	// Permissions for output directory
	OutDirPermission os.FileMode

	// Attach in promiscuous mode for live capture
	Promisc bool

	// Controls whether packets that had an unknown layer will get written into a separate file
	WriteUnknownPackets bool

	// Resolver configuration
	ResolverConfig resolvers.Config

	// Decoding options for gopacket
	DecodeOptions gopacket.DecodeOptions

	// Enable deep packet inspection
	DPI bool

	// Use TCP reassembly
	ReassembleConnections bool

	// LogErrors will log verbose packet decoding errors into the errors.log file
	LogErrors bool

	// NoPrompt will disable all human interaction prompts
	NoPrompt bool

	// HTTPShutdownEndpoint will run a HTTP service on localhost:60589
	// sending a GET request there can be used to trigger teardown and audit record flushing
	// which can be used as alternative to using OS signals
	HTTPShutdownEndpoint bool

	// Timeout for live capture
	// if you set this to 0, the pcap.BlockForever option will be used
	// From the macOS docs on libpcap:
	//   The read timeout is used to arrange that the read not necessarily return
	//   immediately when a packet is seen, but that it wait for some amount of time
	//   to allow more packets to arrive and to read multiple packets from the OS
	//   kernel in one operation.
	Timeout time.Duration

	// Labels is a filesystem path to the labels file on disk
	// that contains the attack mappings
	Labels string

	// Generate scatter chart for the applied labels during labeling.
	Scatter bool

	// ScatterDuration is the interval for data used in the scatter plot.
	ScatterDuration time.Duration
}

Config contains configuration parameters for the Collector instance. this structure has an optimized field order to avoid excessive padding.