infer

README

infer

Infer is a Go package for running predicitions in TensorFlow models.

Overview

This package provides abstractions for running inferences in TensorFlow models for common types. At the moment it only has methods for images, however in the future it can certainly support more.

Getting Started

The easiest way to get going is looking at some examples, two have been provided:

1. Image Recognition API using Inception.
2. MNIST

Setup

This package requires

• Go
• TensorFlow for Go

Installation instructions can be found here. Additionally, a Dockerfile has been included which can be used to run the examples.

Usage

Overview

To use infer, a TensorFlow Graph is required, as well as a defined Input and Output.

Classes may also be included, a slice of possible values. It's assumed the results of any model execution refer to these classes, in order. (e.g. 0 -> mountain, 1 -> cat, 2 -> apple).

m, _ = infer.New(&infer.Model{
  Graph: graph,
  Classes: classes,
  Input: &infer.Input{
    Key:        "input",
    Dimensions: []int32{100, 100},
  },
  Output: &infer.Output{
    Key: "output",
  },
})

Once a new model is defined, inferences can be executed.

predictions, _ := m.FromImage(file, &infer.ImageOptions{})

Predictions are returned sorted by score (most accurate first). A Prediction looks like

Prediction{
  Class: "mountain",
  Score: 0.97,
}

Graph

An infer.Model requires a tf.Graph. The Graph defines the computations required to determine an output based on a provided input. The Graph can be included in two ways:

1. Create the Graph using Go in your application.
2. Load an existing model.

For the latter, an existing model (containing the graph and weights) can be loaded using Go:

model, _ := ioutil.ReadFile("/path/to/model.pb")
graph := tf.NewGraph()
graph.Import(model, "")

For more information on TensorFlow model files, see here.

Input & Output

infer.Input and infer.Output describe TensorFlow layers. In practice this is the layer the input data should be fed to and the layer from which to fetch results.

Each require a Key. This is the unique identifier (name) in the TensorFlow graph for that layer. To see a list of layers and type, the following can be run:

ops := graph.Operations()
for _, o := range ops {
  log.Println(o.Name(), o.Type())
}

If you're not using a pre-trained model, the layers can be named, which can ease in identifying the appropriate layers.

Analyzing Results

In the MNIST example, we can execute a prediction and inspect results as so:

predictions, err := m.FromImage(img, opts)
if err != nil {
  panic(err)
}

// predictions[0].Class -> 8

Documentation

Overview

Package infer provides functions and utilities for running inferences on TensorFlow models.

Index

• Variables
• type ImageOptions
• type Input
• type Model
  • func New(model *Model) (*Model, error)
  • func (m *Model) FromImage(r io.Reader, opts *ImageOptions) ([]*Prediction, error)
  • func (m *Model) FromImageWithContext(ctx context.Context, r io.Reader, opts *ImageOptions) ([]*Prediction, error)
• type Output
• type Prediction
• type Predictions
  • func (ps Predictions) Len() int
  • func (ps Predictions) Less(i, j int) bool
  • func (ps Predictions) Swap(i, j int)

Examples

• Model.FromImage
• New

Variables

var (
	// ErrValidInputOutputRequired occurs if an invalid Input or Output is provided.
	ErrValidInputOutputRequired = errors.New("a valid Input/Output is required")
)

Types

type ImageOptions

type ImageOptions struct {
	// IsGray represents whether the Model expects the input image
	// to be grayscale or not. Specifically, whether the image has
	// 3 channels or 1 channel.
	IsGray bool
}

ImageOptions represent configurable options when evaluating images. Note: for now it is sparse, but included to keep the method signature consistent as new options become available.

type Input

type Input struct {
	// Key represents a layer in a TensorFlow model. The selection of a Key
	// determines "where" the Input/Output occurs in the Graph.
	Key string

	// Dimensions represents the size of the input. It can be of any type but
	// must contains the values expected by the layer. It may be used to
	// augment or resize the input so that it conforms to the specified layer.
	Dimensions interface{}
}

Input is an ML layer. It is identified by a key and has dimensions The dimensions are used to augment or resize the output as appropriate.

type Model

type Model struct {
	Graph   *tf.Graph
	Classes []string
	Input   *Input
	Output  *Output
}

Model is an ML model. It's composed of a computation graph and an Input and Output. It provides methods for running inferences usnig it's Graph, abiding by it's Input/Output.

func New

func New(model *Model) (*Model, error)

New returns a new Model.

Example

package main

import (
	"bytes"
	"io/ioutil"

	"github.com/sjkaliski/infer"
	tf "github.com/tensorflow/tensorflow/tensorflow/go"
)

var m *infer.Model

func main() {
	model, err := ioutil.ReadFile("/path/to/model.pb")
	if err != nil {
		panic(err)
	}

	graph := tf.NewGraph()
	err = graph.Import(model, "")
	if err != nil {
		panic(err)
	}

	m, _ = infer.New(&infer.Model{
		Graph: graph,
		Input: &infer.Input{
			Key:        "input",
			Dimensions: []int32{100, 100},
		},
		Output: &infer.Output{
			Key: "output",
		},
	})
}

func (*Model) FromImage

func (m *Model) FromImage(r io.Reader, opts *ImageOptions) ([]*Prediction, error)

FromImage evaluates an image.

Example

package main

import (
	"bytes"
	"log"

	"github.com/sjkaliski/infer"
)

var (
	m   *infer.Model
	buf bytes.Buffer
)

func main() {
	results, err := m.FromImage(&buf, &infer.ImageOptions{
		IsGray: false,
	})
	if err != nil {
		panic(err)
	}
	log.Println(results)
}

func (*Model) FromImageWithContext

func (m *Model) FromImageWithContext(ctx context.Context, r io.Reader, opts *ImageOptions) ([]*Prediction, error)

FromImageWithContext evaluates an image with context. Optional ImageOptions can be included to dictate the pre-processing of the input image. The method returns an interface of results which can be cast to the appropriate type.

type Output

type Output struct {
	// Key represents a layer in a TensorFlow model. The selection of a Key
	// determines "where" the Input/Output occurs in the Graph.
	Key string

	// Dimensions represents the size of the input. It can be of any type but
	// must contains the values expected by the layer. It may be used to
	// augment or resize the input so that it conforms to the specified layer.
	Dimensions interface{}
}

Output is an ML layer. It is identified by a key and has dimensions The dimensions are used to augment or resize the output as appropriate.

type Prediction

type Prediction struct {
	Class interface{}
	Score float32
}

Prediction represents a class and it's associated score.

type Predictions

type Predictions []*Prediction

Predictions is a list of Prediction.

func (Predictions) Len

func (ps Predictions) Len() int

func (Predictions) Less

func (ps Predictions) Less(i, j int) bool

func (Predictions) Swap

func (ps Predictions) Swap(i, j int)