graphpipe

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 16, 2018 License: UPL-1.0 Imports: 23 Imported by: 0

README

graphpipe-go - GraphPipe for go

graphpipe-go provides a variety of functions to help you easily serve and access ml models using the very speedy GraphPipe protocol.

Additionally, this package provides reference server implementations for common ML model formats, including:

For an overview of GraphPipe, read our project documentation

If you are interested in learning a bit more about how the go servers and clients work, read on:

Client API

Most users integrating GraphPipe into their application will be interested making client calls. To make remote calls, we provide three different APIs

Remote
// Remote is the simple interface for making a remote model request.
// It performs introspection and automatic type conversion on its inputs and outputs.
// Optionally, you can specify inputName and outputName; if either of these
// are missing it is up to the server to infer sane defaults for inputNames
// and outputNames;
func Remote(client *http.Client, uri string, in interface{}, inputName, outputName string) (interface{}, error) {}
MultiRemote
// MultiRemote is a simple interface for communicating with models
// that have multiple inputs and outputs.  It is recommended that you
// Specify inputNames and outputNames so that you can control input/output
// ordering.  MultiRemote also performs type introspection for inputs and outputs.
func MultiRemote(client *http.Client, uri string, ins []interface{}, inputNames, outputNames []string) ([]interface{}, error) {}
MultiRemoteRaw
// MultiRemoteRaw is the actual implementation of the remote model
// request using NativeTensor objects.
func MultiRemoteRaw(client *http.Client, uri string, inputs []*NativeTensor, inputNames, outputNames []string) ([]*NativeTensor, error) {}

In similar fashion to the serving model, the client for making remote calls is made up of three functions, Remote, MultiRemote, and MultiRemoteRaw.

The first two of those will convert your native Go types into tensors and back, while the last one uses graphpipe tensors throughout.

Model Serving API

There are two Serve functions, Serve and ServeRaw, that both create standard Go http listeners and support caching with BoltDB.

Serve

For applications where the manipulation of tensors will mostly be in go, Serve provides a wrapper to your apply function that allows you to work with standard go types rather than explicit graphpipe data objects and converts between them for you. From the Go docs:

// Serve offers multiple inputs and outputs and converts tensors
// into native datatypes based on the shapes passed in to this function
// plus any additional shapes implied by your apply function.
// If cache is true, will attempt to cache using cache.db as cacheFile
func Serve(listen string, cache bool, apply interface{}, inShapes, outShapes [][]int64) error {}

As an example, here is a simple way to construct a graphpipe identity server, which can receive a graphpipe network request, and echo it back to the client:

package main

import (
    "github.com/Sirupsen/logrus"
    graphpipe "github.com/oracle/graphpipe-go"
)

func main() {
    logrus.SetLevel(logrus.InfoLevel)
    useCache := false           // toggle caching on/off
    inShapes := [][]int64(nil)  // Optionally set input shapes
    outShapes := [][]int64(nil) // Optionally set output shapes
    if err := graphpipe.Serve("0.0.0.0:9000", useCache, apply, inShapes, outShapes); err != nil {
        logrus.Errorf("Failed to serve: %v", err)
    }
}

func apply(requestContext *graphpipe.RequestContext, ignore string, in interface{}) interface{} {
    return in // using the graphpipe.Serve interface, graphpipe automatically converts
              // go native types to tensors.
}

You can find a docker-buildable example of this server here.

ServeRaw

For applications that will be passing the tensors directly to another system for processing and don't need conversion to standard Go types, ServeRaw provides a lower-level interface. For examples of apps that use ServeRaw, see graphpipe-tf and graphpipe-onnx.

As you might expect, Serve uses ServeRaw underneath the hood.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func BuildDataTensorRaw 

func BuildDataTensorRaw(b *fb.Builder, data []byte, shape []int64, dt uint8) fb.UOffsetT

BuildDataTensorRaw builds a data tensor from a byte slice. Validity is not checked, so passing a data slice that is not shape.numElems * dt.size will result in a tensor that is unusable by the receiver.

func BuildStringTensorRaw 

func BuildStringTensorRaw(b *fb.Builder, strs []string, shape []int64) fb.UOffsetT

BuildStringTensorRaw builds a string tensor from a string slice.

func BuildTensorContiguous 

func BuildTensorContiguous(b *fb.Builder, val interface{}) (fb.UOffsetT, error)

BuildTensorContiguous builds a flatbuffer tensor from a native go slice or array. Using this on a non-contiguous slice or a jagged slice can result in memory corruption. Note that this returns a flatbuffers offset type so that it can be used as part of building a larger object. The object can be serialized with the Serialize function.

func BuildTensorNonContiguous 

func BuildTensorNonContiguous(b *fb.Builder, val interface{}) (fb.UOffsetT, error)

BuildTensorNonContiguous builds a flatbuffer tensor from a native go slice. This should be used if the slice is not contiguous in memory. Using this on a jagged slice can result in memory corruption. Note that this returns a flatbuffers offset type so that it can be used as part of building a larger object. The object can be serialized with the Serialize function.

func BuildTensorSafe 

func BuildTensorSafe(b *fb.Builder, val interface{}) (fb.UOffsetT, error)

BuildTensorSafe builds a flatbuffer tensor from a native go slice or array. This version is safe to use with any array or slice. It will return an error if val contains a jagged slice. Note that this returns a flatbuffers offset type so that it can be used as part of building a larger object. The object can be serialized with the Serialize function.

func Handler 

func Handler(c *appContext, w http.ResponseWriter, r *http.Request) error

Handler handles our http requests.

func ListenAndServe 

func ListenAndServe(addr string, handler http.Handler) error

ListenAndServe is like robocop but for servers (listens on a host:port and handles requests).

func MultiRemote 

func MultiRemote(client *http.Client, uri string, ins []interface{}, inputNames, outputNames []string) ([]interface{}, error)

MultiRemote is a simple interface for communicating with models that have multiple inputs and outputs. It is recommended that you Specify inputNames and outputNames so that you can control input/output ordering. MultiRemote also performs type introspection for inputs and outputs.

func NativeTensorToNative 

func NativeTensorToNative(t *NativeTensor) (interface{}, error)

NativeTensorToNative is a converter between NativeTensors and raw arrays of arrays (of arrays of arrays) of numbers.

func Remote 

func Remote(client *http.Client, uri string, in interface{}, inputName, outputName string) (interface{}, error)

Remote is the simple interface for making a remote model request. It performs introspection and automatic type conversion on its inputs and outputs. Optionally, you can specify inputName and outputName; if either of these are missing it is up to the server to infer sane defaults for inputNames and outputNames;

func Serialize 

func Serialize(b *fb.Builder, obj fb.UOffsetT) []byte

Serialize writes a builder object to a byte array

func Serve 

func Serve(listen string, cache bool, apply interface{}, inShapes, outShapes [][]int64) error

Serve offers multiple inputs and outputs and converts tensors into native datatypes based on the shapes passed in to this function plus any additional shapes implied by your apply function. If cache is true, will attempt to cache using cache.db as cacheFile

func ServeRaw 

func ServeRaw(opts *ServeRawOptions) error

ServeRaw starts the model server. The listen address and port can be specified with the listen parameter. If cacheFile is not "" then caches will be stored using it. context will be passed back to the handler

func ShapeType 

func ShapeType(val reflect.Value) (shape []int64, num int64, size int64, dt uint8, err error)

ShapeType returns shape, num, size, and dt for a reflect.Value. The value of size will be -1 (unknown) for a string type

func TensorToNative 

func TensorToNative(t *graphpipefb.Tensor) (interface{}, error)

TensorToNative converts a tensor object into a native go slice. It can be cast to a proper type. For example: x := TensorToNative(t).([][]float32)

Types

type Applier 

type Applier func(*RequestContext, string, map[string]*NativeTensor, []string) ([]*NativeTensor, error)

Applier is the base signature for server actions.

type Error 

type Error interface {
	error
	Status() int
}

Error is our wrapper around the error interface.

type GetHandlerFunc 

type GetHandlerFunc func(http.ResponseWriter, *http.Request, []byte) error

GetHandlerFunc is an indirection to return the handler.

type NativeIOMetadata 

type NativeIOMetadata struct {
	Name        string
	Description string
	Shape       []int64
	Type        uint8
}

NativeIOMetadata holds information describing the format of the model being served.

type NativeMetadataResponse 

type NativeMetadataResponse struct {
	Name        string
	Version     string
	Server      string
	Description string
	Inputs      []NativeIOMetadata
	Outputs     []NativeIOMetadata
}

NativeMetadataResponse is the response format used by the server.

func (*NativeMetadataResponse) Build 

func (meta *NativeMetadataResponse) Build(b *fb.Builder) fb.UOffsetT

Build does all the heavy lifting of building out flatbuffers.

type NativeTensor 

type NativeTensor struct {
	Type       uint8
	Shape      []int64
	StringVals []string
	Data       []byte
}

NativeTensor is an easier to use version of the flatbuffer Tensor.

func MultiRemoteRaw 

func MultiRemoteRaw(client *http.Client, uri string, inputs []*NativeTensor, inputNames, outputNames []string) ([]*NativeTensor, error)

MultiRemoteRaw is the actual implementation of the remote model request using NativeTensor objects.

func TensorToNativeTensor 

func TensorToNativeTensor(t *graphpipefb.Tensor) *NativeTensor

TensorToNativeTensor is a converter between the flatbuffer Tensor objects and the easier to use NativeTensor objects.

func (*NativeTensor) Build 

func (nt *NativeTensor) Build(b *fb.Builder) fb.UOffsetT

Build creates the actual flatbuffer representation.

func (*NativeTensor) InitSimple 

func (nt *NativeTensor) InitSimple(val interface{}) error

InitSimple tries to infer your data shape from the value you give it.

func (*NativeTensor) InitWithData 

func (nt *NativeTensor) InitWithData(data []byte, shape []int64, dt uint8) error

InitWithData is a more explicit initialization and expects the data to already be in the correct format.

func (*NativeTensor) InitWithStringVals 

func (nt *NativeTensor) InitWithStringVals(stringVals []string, shape []int64) error

InitWithStringVals is a more explicit initialization and expects the data to already be in the correct format (for stringvals).

type Nt 

type Nt struct {
	// contains filtered or unexported fields
}

Nt is a NativeTensor holding struct.

type RequestContext 

type RequestContext struct {
	CleanupFunc func()
	// contains filtered or unexported fields
}

RequestContext attaches our flatbuffers to the request.

func (*RequestContext) BuildTensor 

func (ctx *RequestContext) BuildTensor(val interface{}) (fb.UOffsetT, error)

BuildTensor does the heavy lifting to make sure we have a flatbuffer.

func (*RequestContext) IsAlive 

func (ctx *RequestContext) IsAlive() bool

IsAlive tells you if it isn't dead.

func (*RequestContext) SetDead 

func (ctx *RequestContext) SetDead()

SetDead makes sure it isn't alive.

type ServeRawOptions 

type ServeRawOptions struct {
	Listen         string
	CacheFile      string
	Meta           *NativeMetadataResponse
	DefaultInputs  []string
	DefaultOutputs []string
	Apply          Applier
	GetHandler     GetHandlerFunc
}

ServeRawOptions is just a call parameter struct.

func BuildSimpleApply 

func BuildSimpleApply(apply interface{}, inShapes, outShapes [][]int64) *ServeRawOptions

BuildSimpleApply is the factory for producing SimpleAppliers

type SimpleApplier 

type SimpleApplier func(interface{}) (interface{}, error)

SimpleApplier is the signature for a server action that converts between native types and graphpipe objects.

type StatusError 

type StatusError struct {
	Code int
	Err  error
}

StatusError is our wrapper around an http error interface.

func (StatusError) Error 

func (se StatusError) Error() string

Error returns the error message.

func (StatusError) Status 

func (se StatusError) Status() int

Status returns an http status code.

Source Files

View all Source files

Directories

Path Synopsis
cmd
graphpipe-echo
graphpipe-onnx
graphpipe-tf
graphpipe-tf/internal/github.com/tensorflow/tensorflow/tensorflow/go/core/framework
Package framework is a generated protocol buffer package.
 Package framework is a generated protocol buffer package.
graphpipe-tf/internal/github.com/tensorflow/tensorflow/tensorflow/go/core/protobuf
Package protobuf is a generated protocol buffer package.
 Package protobuf is a generated protocol buffer package.
graphpipe-tf/protoc-gen-gogotf

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.