caffe2

README

Go Bindings for Caffe2

Caffe2 Installation

Please refer to scripts or the LIBRARY INSTALLATION section in the dockefiles to install Caffe2 on your system. OpenBLAS is used.

If you get an error about not being able to write to /opt then perform the following

sudo mkdir -p /opt/pytorch
sudo chown -R `whoami` /opt/pytorch

• The default blas is OpenBLAS. The default OpenBLAS path for mac os is /usr/local/opt/openblas if installed throught homebrew (openblas is keg-only, which means it was not symlinked into /usr/local, because macOS provides BLAS and LAPACK in the Accelerate framework).

• The default Caffe2 installation path is /opt/pytorch/caffe2.

• The default CUDA path is /usr/local/cuda

See lib.go for details.

After installing Caffe2, run export DYLD_LIBRARY_PATH=/opt/pytorch/caffe2/lib:$DYLD_LIBRARY_PATH on mac os or export LD_LIBRARY_PATH=/opt/pytorch/caffe2/lib:$DYLD_LIBRARY_PATHon linux.

Use Other Libary Paths

To use different library paths, change CGO_CFLAGS, CGO_CXXFLAGS and CGO_LDFLAGS enviroment variables.

For example,

    export CGO_CFLAGS="${CGO_CFLAGS} -I /usr/local/cuda-9.2/include -I/usr/local/cuda-9.2/nvvm/include -I /usr/local/cuda-9.2/extras/CUPTI/include -I /usr/local/cuda-9.2/targets/x86_64-linux/include -I /usr/local/cuda-9.2/targets/x86_64-linux/include/crt"
    export CGO_CXXFLAGS="${CGO_CXXFLAGS} -I /usr/local/cuda-9.2/include -I/usr/local/cuda-9.2/nvvm/include -I /usr/local/cuda-9.2/extras/CUPTI/include -I /usr/local/cuda-9.2/targets/x86_64-linux/include -I /usr/local/cuda-9.2/targets/x86_64-linux/include/crt"
    export CGO_LDFLAGS="${CGO_LDFLAGS} -L /usr/local/nvidia/lib64 -L /usr/local/cuda-9.2/nvvm/lib64 -L /usr/local/cuda-9.2/lib64 -L /usr/local/cuda-9.2/lib64/stubs -L /usr/local/cuda-9.2/targets/x86_64-linux/lib/stubs/ -L /usr/local/cuda-9.2/lib64/stubs -L /usr/local/cuda-9.2/extras/CUPTI/lib64"

Run go build in to check the Caffe2 installation and library paths set-up.

Run Then Examples

Make sure you have already install mlmodelscope dependences and set up the external services.

On linux, the default is to use GPU, if you don't have a GPU, do go build -tags nogpu instead of go build.

batch

This example is to show how to use mlmodelscope tracing to profile the inference.

  cd example/batch
  go build
  ./batch

Then you can go to localhost:16686 to look at the trace of that inference.

batch_nvprof

You need GPU and CUDA to run this example. This example is to show how to use nvprof to profile the inference.

  cd example/batch_nvprof
  go build
  nvprof --profile-from-start off ./batch_nvprof

Refer to Profiler User's Guide on how to use nvprof.

Documentation

Index

• Variables
• type Device
• type Predictor
  • func New(ctx context.Context, opts ...options.Option) (*Predictor, error)
  • func (p *Predictor) Close()
  • func (p *Predictor) DisableProfiling() error
  • func (p *Predictor) EndProfiling() error
  • func (p *Predictor) Predict(ctx context.Context, data []float32, channels int, width int, height int) error
  • func (p *Predictor) ReadPredictionOutput(ctx context.Context) ([]float32, error)
  • func (p *Predictor) ReadProfile() (string, error)
  • func (p *Predictor) StartProfiling(name, metadata string) error

Variables

var (
	Version   = "0.4.1"
	BuildDate = "undefined"
	GitCommit = "undefined"
)

Types

type Device

type Device int

const (
	CPUDevice  Device = Device(C.CPU_DEVICE_KIND)
	CUDADevice        = Device(C.CUDA_DEVICE_KIND)
)

type Predictor

type Predictor struct {
	// contains filtered or unexported fields
}

func New

func New(ctx context.Context, opts ...options.Option) (*Predictor, error)

func (*Predictor) Close

func (p *Predictor) Close()

func (*Predictor) DisableProfiling

func (p *Predictor) DisableProfiling() error

func (*Predictor) EndProfiling

func (p *Predictor) EndProfiling() error

func (*Predictor) Predict

func (p *Predictor) Predict(ctx context.Context, data []float32, channels int,
	width int, height int) error

func (*Predictor) ReadPredictionOutput

func (p *Predictor) ReadPredictionOutput(ctx context.Context) ([]float32, error)

func (*Predictor) ReadProfile

func (p *Predictor) ReadProfile() (string, error)

func (*Predictor) StartProfiling

func (p *Predictor) StartProfiling(name, metadata string) error