nnaccel

README

NN Accelerator

This package dynamically loads a .so that implements neural network inference. We do it like this so that our Go binary doesn't have a hard dependency on whatever libraries are needed for, eg, the Hailo NPUs.

If there was a way to avoid this dynamic binary loading infrastructure, I would do it!

Caveats

The Hailo models require the NMS thresholds to be set at compile time. Our NN interface allows one to specify it at runtime. This is a problem, and will cause confusion. I haven't decided how to deal with this yet.

Documentation

Index

• func CError(cerr *C.char) error
• func PageAlignedAlloc(size int) []byte
• func PageSize() int
• func RoundUpToPageSize(size int) int
• type Accelerator
  • func Load(accelName string) (*Accelerator, error)
  • func (m *Accelerator) OpenDevice() (*Device, error)
  • func (m *Accelerator) StatusToErr(status C.int) error
• type AsyncJob
  • func (j *AsyncJob) Close()
  • func (j *AsyncJob) GetObjectDetections(batchEl int) ([]nn.ObjectDetection, error)
  • func (j *AsyncJob) Wait(wait time.Duration) bool
• type Device
  • func (d *Device) Close()
  • func (d *Device) LoadModel(filename string, setup *nn.ModelSetup) (*Model, error)
  • func (d *Device) ModelFiles() (subdir string, ext []string)
• type Model
  • func (m *Model) Close()
  • func (m *Model) Config() *nn.ModelConfig
  • func (m *Model) DetectObjects(batch nn.ImageBatch, params *nn.DetectionParams) ([][]nn.ObjectDetection, error)
  • func (m *Model) Run(batchSize, batchStride, width, height, nchan int, stride int, ...) (*AsyncJob, error)

Functions

func CError

func CError(cerr *C.char) error

Consume a C heap allocated char* and return it as a Go error. Before returning, free the C char*. If the input is NULL, then return nil.

func PageAlignedAlloc

func PageAlignedAlloc(size int) []byte

Allocate 'size' bytes of memory, aligned to a page boundary.

func PageSize

func PageSize() int

Returns the system page size

func RoundUpToPageSize

func RoundUpToPageSize(size int) int

Round size up to the nearest page size

Types

type Accelerator

type Accelerator struct {
	// contains filtered or unexported fields
}

func Load

func Load(accelName string) (*Accelerator, error)

Load an NN accelerator. At present, the only accelerator we have is "hailo"

func (*Accelerator) OpenDevice

func (m *Accelerator) OpenDevice() (*Device, error)

Open a new device (eg a handle to a GPU or a Hailo accelerator) A device must be closed after using.

func (*Accelerator) StatusToErr

func (m *Accelerator) StatusToErr(status C.int) error

type AsyncJob

type AsyncJob struct {
	// contains filtered or unexported fields
}

func (*AsyncJob) Close

func (j *AsyncJob) Close()

func (*AsyncJob) GetObjectDetections

func (j *AsyncJob) GetObjectDetections(batchEl int) ([]nn.ObjectDetection, error)

func (*AsyncJob) Wait

func (j *AsyncJob) Wait(wait time.Duration) bool

Returns true if the job is finished

type Device

type Device struct {
	// contains filtered or unexported fields
}

func (*Device) Close

func (d *Device) Close()

Close a device

func (*Device) LoadModel

func (d *Device) LoadModel(filename string, setup *nn.ModelSetup) (*Model, error)

func (*Device) ModelFiles

func (d *Device) ModelFiles() (subdir string, ext []string)

type Model

type Model struct {
	// contains filtered or unexported fields
}

func (*Model) Close

func (m *Model) Close()

func (*Model) Config

func (m *Model) Config() *nn.ModelConfig

func (*Model) DetectObjects

func (m *Model) DetectObjects(batch nn.ImageBatch, params *nn.DetectionParams) ([][]nn.ObjectDetection, error)

Detection thresholds are ignored here. They need to be setup when the model is initially loaded.

func (*Model) Run

func (m *Model) Run(batchSize, batchStride, width, height, nchan int, stride int, images unsafe.Pointer) (*AsyncJob, error)