go_dpi

README

go-dpi

go-dpi is an open source Go library for application layer protocol identification of traffic flows. In addition to its own heuristic methods, it contains wrappers for other popular and well-established libraries that also perform protocol identification, such as nDPI and libprotoident. It aims to provide a simple, easy-to-use interface and the capability to be extended by a developer with new detection methods and protocols.

It attempts to classify flows to different protocols regardless of the ports used. This makes it possible to detect protocols on non-standard ports, which is ideal for honeypots, as malware might often try and throw off detection methods by using non-standard and unregistered ports. Also, with its layered architecture, it aims to be fast in its detection, only using heavier classification methods when the faster ones fail.

It is being developed in the context of the Google Summer of Code 2017 program, under the mentorship of The Honeynet Project.

Please read the project's Wiki page for more information.

For documentation, please check out the godoc reference.

Example usage

The library and the modules APIs aim to be very simple and straightforward to use. The library relies on the gopacket library and its Packet structure. Once you have a Packet in your hands, it's very easy to classify it with the library. First of all you need to initialize the library. You can do that by calling:

godpi.Initialize()

The Initialize method initializes all the selected modules in the library, by calling the Initialize method that they provide. It also creates the cache that is used to track the flows, which outdates unused flows after some minutes.

Then, you need a flow that contains the packet. You can get the flow a packet belongs to with the following call:

flow, isNew := godpi.GetPacketFlow(packet)

That call returns the flow, as well as whether that flow is a new one (this packet is the first in the flow) or an existing one.

Afterwards, classifying the flow can be done by calling:

result := godpi.ClassifyFlow(flow)

This returns the protocol guessed by the classifiers as well as the source, e.g. go-dpi or one of the wrappers.

Finally, once you are done with the library, you should free the used resources by calling:

godpi.Destroy()

Destroy frees all the resources that the library is using, and calls the Destroy method of all the activated modules. It is essentially the opposite of the Initialize method.

A minimal example application is included below. It uses the library to classify a packet capture file, located at /tmp/http.cap. Note the helpful godpi.ReadDumpFile function that returns a channel with all the packets in the file.

package main

import (
	"fmt"
	"github.com/GreyNoise-Intelligence/go-dpi"
	"github.com/GreyNoise-Intelligence/go-dpi/types"
	"github.com/GreyNoise-Intelligence/go-dpi/utils"
)

func main() {
	godpi.Initialize()
	defer godpi.Destroy()
	packets, err := utils.ReadDumpFile("/tmp/http.cap")
	if err != nil {
		fmt.Println(err)
	} else {
		for packet := range packets {
			flow, _ := godpi.GetPacketFlow(packet)
			result := godpi.ClassifyFlow(flow)
			if result.Protocol != types.Unknown {
				fmt.Println(result.Source, "detected protocol", result.Protocol)
			} else {
				fmt.Println("No detection was made")
			}
		}
	}
}

License

go-dpi is available under the MIT license and distributed in source code format.

Documentation

Overview

Package godpi provides the main API interface for utilizing the go-dpi library.

Package godpi provides the main API interface for utilizing the go-dpi library.

Index

• func ClassifyFlow(flow *types.Flow) (result types.ClassificationResult)
• func ClassifyFlowAllModules(flow *types.Flow) (results []types.ClassificationResult)
• func Destroy() (errs []error)
• func GetPacketFlow(packet gopacket.Packet) (*types.Flow, bool)
• func Initialize(opts ...Options) (errs []error)
• func SetCacheExpiration(expiration time.Duration)
• func SetModules(modules []types.Module)
• type ClassifierOption
  • func (o ClassifierOption) Apply(mod types.Module)
• type Options

Functions

func ClassifyFlow

func ClassifyFlow(flow *types.Flow) (result types.ClassificationResult)

ClassifyFlow takes a Flow and tries to classify it with all of the activated modules in order, until one of them manages to classify it. It returns the detected protocol as well as the source that made the classification. If no classification is made, the protocol Unknown is returned.

func ClassifyFlowAllModules

func ClassifyFlowAllModules(flow *types.Flow) (results []types.ClassificationResult)

ClassifyFlowAllModules takes a Flow and tries to classify it with all of the activated modules. However, as opposed to ClassifyFlow, it will return all of the results returned from the modules, not only the first successful one.

func Destroy

func Destroy() (errs []error)

Destroy frees all allocated resources and deactivates the active modules.

func GetPacketFlow

func GetPacketFlow(packet gopacket.Packet) (*types.Flow, bool)

GetPacketFlow returns a Flow for the given packet. If another packet has been processed before that was part of the same communication flow, the same Flow will be returned, with the new packet added. Otherwise, a new Flow will be created with only this packet. The function also returns whether the returned Flow is a new one, and not one that already existed.

func Initialize

func Initialize(opts ...Options) (errs []error)

Initialize initializes the library and the selected modules.

func SetCacheExpiration

func SetCacheExpiration(expiration time.Duration)

SetCacheExpiration sets how long after being inactive flows should be discarded from the flow tracker. If a negative value is passed, flows will never expire. By default, this value is 5 minutes. After calling this method, Initialize should be called, in order to initialize the cache. If Initialize has already been called before, Destroy should be called as well before Initialize.

func SetModules

func SetModules(modules []types.Module)

SetModules selects the modules to be used by the library and their order. After calling this method, Initialize should be called, in order to initialize any new modules. If Initialize has already been called before, Destroy should be called as well before Initialize.

Types

type ClassifierOption

type ClassifierOption struct {
}

ClassifierOption take classifier options to override default values for now this option was added for test

func (ClassifierOption) Apply

func (o ClassifierOption) Apply(mod types.Module)

type Options

type Options interface {
	Apply(types.Module)
}

Options allow end users init module with custom options NOTE. it's necessary to check the module passed in Apply func