vision

Documentation

Overview

Package vision is the service that allows you to access various computer vision algorithms (like detection, segmentation, tracking, etc) that usually only require a camera or image input.

Index

• Constants
• Variables
• func FindFirstName(r robot.Robot) string
• func Named(name string) resource.Name
• func NewServer(s subtype.Service) pb.VisionServiceServer
• func NewUnimplementedInterfaceError(actual interface{}) error
• func WrapWithReconfigurable(s interface{}, name resource.Name) (resource.Reconfigurable, error)
• type Attributes
  • func (a *Attributes) Walk(visitor config.Visitor) (interface{}, error)
• type Service
  • func FirstFromLocalRobot(r robot.Robot) (Service, error)
  • func FirstFromRobot(r robot.Robot) (Service, error)
  • func FromRobot(r robot.Robot, name string) (Service, error)
  • func NewClientFromConn(ctx context.Context, conn rpc.ClientConn, name string, logger golog.Logger) Service
• type VisModelConfig
• type VisModelType

Constants

const SubtypeName = resource.SubtypeName("vision")

SubtypeName is the name of the type of service.

Variables

var Subtype = resource.NewSubtype(
	resource.ResourceNamespaceRDK,
	resource.ResourceTypeService,
	SubtypeName,
)

Subtype is a constant that identifies the vision service resource subtype.

Functions

func FindFirstName

func FindFirstName(r robot.Robot) string

FindFirstName returns name of first vision service found.

func Named

func Named(name string) resource.Name

Named is a helper for getting the named vision's typed resource name. RSDK-347 Implements vision's Named.

func NewServer

func NewServer(s subtype.Service) pb.VisionServiceServer

NewServer constructs a vision gRPC service server.

func NewUnimplementedInterfaceError

func NewUnimplementedInterfaceError(actual interface{}) error

NewUnimplementedInterfaceError is used when there is a failed interface check.

func WrapWithReconfigurable

func WrapWithReconfigurable(s interface{}, name resource.Name) (resource.Reconfigurable, error)

WrapWithReconfigurable wraps a vision service as a Reconfigurable.

Types

type Attributes

type Attributes struct {
	ModelRegistry []VisModelConfig `json:"register_models"`
}

Attributes contains a list of the user-provided details necessary to register a new vision service.

func (*Attributes) Walk

func (a *Attributes) Walk(visitor config.Visitor) (interface{}, error)

Walk implements the config.Walker interface.

type Service

type Service interface {
	// model parameters
	GetModelParameterSchema(ctx context.Context, modelType VisModelType, extra map[string]interface{}) (*jsonschema.Schema, error)
	// detector methods
	DetectorNames(ctx context.Context, extra map[string]interface{}) ([]string, error)
	AddDetector(ctx context.Context, cfg VisModelConfig, extra map[string]interface{}) error
	RemoveDetector(ctx context.Context, detectorName string, extra map[string]interface{}) error
	DetectionsFromCamera(ctx context.Context, cameraName, detectorName string, extra map[string]interface{}) ([]objdet.Detection, error)
	Detections(ctx context.Context, img image.Image, detectorName string, extra map[string]interface{}) ([]objdet.Detection, error)
	// classifier methods
	ClassifierNames(ctx context.Context, extra map[string]interface{}) ([]string, error)
	AddClassifier(ctx context.Context, cfg VisModelConfig, extra map[string]interface{}) error
	RemoveClassifier(ctx context.Context, classifierName string, extra map[string]interface{}) error
	ClassificationsFromCamera(
		ctx context.Context,
		cameraName, classifierName string,
		n int,
		extra map[string]interface{},
	) (classification.Classifications, error)
	Classifications(
		ctx context.Context,
		img image.Image,
		classifierName string,
		n int,
		extra map[string]interface{},
	) (classification.Classifications, error)
	// segmenter methods
	SegmenterNames(ctx context.Context, extra map[string]interface{}) ([]string, error)
	AddSegmenter(ctx context.Context, cfg VisModelConfig, extra map[string]interface{}) error
	RemoveSegmenter(ctx context.Context, segmenterName string, extra map[string]interface{}) error
	GetObjectPointClouds(ctx context.Context, cameraName, segmenterName string, extra map[string]interface{}) ([]*viz.Object, error)
	resource.Generic
}

A Service that implements various computer vision algorithms like detection and segmentation.

func FirstFromLocalRobot

func FirstFromLocalRobot(r robot.Robot) (Service, error)

FirstFromLocalRobot returns the first vision service on this main robot. This will specifically ignore remote resources.

func FirstFromRobot

func FirstFromRobot(r robot.Robot) (Service, error)

FirstFromRobot returns the first vision service on this robot.

func FromRobot

func FromRobot(r robot.Robot, name string) (Service, error)

FromRobot is a helper for getting the named vision service from the given Robot.

func NewClientFromConn

func NewClientFromConn(ctx context.Context, conn rpc.ClientConn, name string, logger golog.Logger) Service

NewClientFromConn constructs a new Client from connection passed in.

type VisModelConfig

type VisModelConfig struct {
	Name       string              `json:"name"`
	Type       string              `json:"type"`
	Parameters config.AttributeMap `json:"parameters"`
}

VisModelConfig specifies the name of the detector, the type of detector, and the necessary parameters needed to build the detector.

type VisModelType

type VisModelType string

VisModelType defines what vision models are known by the vision service.