vision

Documentation

Overview

Package vision is the service that allows you to access various computer vision algorithms (like detection, segmentation, tracking, etc) that usually only require a camera or image input.

Index

• Constants
• Variables
• func Named(name string) resource.Name
• func NewServer(s subtype.Service) pb.VisionServiceServer
• func NewTFLiteDetector(ctx context.Context, cfg *DetectorConfig, logger golog.Logger) (objectdetection.Detector, *inf.TFLiteStruct, error)
• func WrapWithReconfigurable(s interface{}) (resource.Reconfigurable, error)
• type Attributes
• type DetectorConfig
• type DetectorType
• type SegmenterRegistration
• type Service
  • func FromRobot(r robot.Robot) (Service, error)
  • func New(ctx context.Context, r robot.Robot, config config.Service, logger golog.Logger) (Service, error)
  • func NewClientFromConn(ctx context.Context, conn rpc.ClientConn, name string, logger golog.Logger) Service
• type TFLiteDetectorConfig

Constants

const (
	TFLiteType     = DetectorType("tflite")
	TensorFlowType = DetectorType("tensorflow")
	ColorType      = DetectorType("color")
)

The set of allowed detector types.

const RadiusClusteringSegmenter = "radius_clustering"

RadiusClusteringSegmenter is the name of a segmenter that finds well separated objects on a flat plane.

const SubtypeName = resource.SubtypeName("vision")

SubtypeName is the name of the type of service.

Variables

var Name = resource.NameFromSubtype(Subtype, "")

Name is the Vision Service's typed resource name.

var Subtype = resource.NewSubtype(
	resource.ResourceNamespaceRDK,
	resource.ResourceTypeService,
	SubtypeName,
)

Subtype is a constant that identifies the vision service resource subtype.

Functions

func Named

func Named(name string) resource.Name

Named is a helper for getting the named vision's typed resource name. RSDK-347 Implements vision's Named.

func NewServer

func NewServer(s subtype.Service) pb.VisionServiceServer

NewServer constructs a vision gRPC service server.

func NewTFLiteDetector

func NewTFLiteDetector(ctx context.Context, cfg *DetectorConfig, logger golog.Logger) (objectdetection.Detector, *inf.TFLiteStruct, error)

NewTFLiteDetector creates an RDK detector given a DetectorConfig. In other words, this function returns a function from image-->[]objectdetection.Detection. It does this by making calls to an inference package and wrapping the result.

func WrapWithReconfigurable

func WrapWithReconfigurable(s interface{}) (resource.Reconfigurable, error)

WrapWithReconfigurable wraps a vision service as a Reconfigurable.

Types

type Attributes

type Attributes struct {
	DetectorRegistry []DetectorConfig `json:"register_detectors"`
}

Attributes contains a list of the user-provided details necessary to register a new vision service.

type DetectorConfig

type DetectorConfig struct {
	Name       string              `json:"name"`
	Type       string              `json:"type"`
	Parameters config.AttributeMap `json:"parameters"`
}

DetectorConfig specifies the name of the detector, the type of detector, and the necessary parameters needed to build the detector.

type DetectorType

type DetectorType string

DetectorType defines what detector types are known.

type SegmenterRegistration

type SegmenterRegistration struct {
	segmentation.Segmenter
	Parameters []utils.TypedName
}

A SegmenterRegistration stores both the Segmenter function and the form of the parameters it takes as an argument.

type Service

type Service interface {
	// detector methods
	GetDetectorNames(ctx context.Context) ([]string, error)
	AddDetector(ctx context.Context, cfg DetectorConfig) error
	GetDetectionsFromCamera(ctx context.Context, cameraName, detectorName string) ([]objdet.Detection, error)
	GetDetections(ctx context.Context, img image.Image, detectorName string) ([]objdet.Detection, error)
	// segmenter methods
	GetSegmenterNames(ctx context.Context) ([]string, error)
	GetSegmenterParameters(ctx context.Context, segmenterName string) ([]utils.TypedName, error)
	GetObjectPointClouds(ctx context.Context, cameraName, segmenterName string, params config.AttributeMap) ([]*viz.Object, error)
}

A Service that implements various computer vision algorithms like detection and segmentation.

func FromRobot

func FromRobot(r robot.Robot) (Service, error)

FromRobot retrieves the vision service of a robot.

func New

func New(ctx context.Context, r robot.Robot, config config.Service, logger golog.Logger) (Service, error)

New registers new detectors from the config and returns a new object detection service for the given robot.

func NewClientFromConn

func NewClientFromConn(ctx context.Context, conn rpc.ClientConn, name string, logger golog.Logger) Service

NewClientFromConn constructs a new Client from connection passed in.

type TFLiteDetectorConfig

type TFLiteDetectorConfig struct {
	// this should come from the attributes part of the detector config
	ModelPath  string  `json:"model_path"`
	NumThreads *int    `json:"num_threads"`
	LabelPath  *string `json:"label_path"`
	ServiceURL *string `json:"service_url"`
}

TFLiteDetectorConfig specifies the fields necessary for creating a TFLite detector.