camera

Documentation

Overview

Package camera defines an image capturing device. For more information, see the camera component docs.

Index

• Constants
• Variables
• func DecodeImageFromCamera(ctx context.Context, mimeType string, extra map[string]interface{}, cam Camera) (image.Image, error)
• func Named(name string) resource.Name
• func NamesFromRobot(r robot.Robot) []string
• func NewPinholeModelWithBrownConradyDistortion(pinholeCameraIntrinsics *transform.PinholeCameraIntrinsics, ...) transform.PinholeCameraModel
• func NewPropertiesError(cameraIdentifier string) error
• func NewRPCServiceServer(coll resource.APIResourceCollection[Camera]) interface{}
• func NewUnsupportedImageTypeError(s ImageType) error
• func ReadImage(ctx context.Context, src gostream.VideoSource) (image.Image, func(), error)
• type Camera
  • func FromDependencies(deps resource.Dependencies, name string) (Camera, error)
  • func FromRobot(r robot.Robot, name string) (Camera, error)
  • func FromVideoSource(name resource.Name, src VideoSource, logger logging.Logger) Camera
  • func NewClientFromConn(ctx context.Context, conn rpc.ClientConn, remoteName string, ...) (Camera, error)
• type ImageMetadata
  • func ReadImageBytes(ctx context.Context, src gostream.VideoSource, mimeType string) ([]byte, ImageMetadata, error)
• type ImageType
• type ImagesSource
• type NamedImage
• type PointCloudSource
• type Properties
• type VideoSource
  • func NewVideoSourceFromReader(ctx context.Context, reader gostream.VideoReader, ...) (VideoSource, error)
  • func WrapVideoSourceWithProjector(ctx context.Context, source gostream.VideoSource, ...) (VideoSource, error)

Constants

const (
	UnspecifiedStream = ImageType("")
	ColorStream       = ImageType("color")
	DepthStream       = ImageType("depth")
)

The allowed types of streams that can come from a VideoSource.

const SubtypeName = "camera"

SubtypeName is a constant that identifies the camera resource subtype string.

Variables

var (
	// ErrNoPeerConnection indicates there was no peer connection.
	ErrNoPeerConnection = errors.New("no PeerConnection")
	// ErrNoSharedPeerConnection indicates there was no shared peer connection.
	ErrNoSharedPeerConnection = errors.New("no Shared PeerConnection")
	// ErrUnknownSubscriptionID indicates that a SubscriptionID is unknown.
	ErrUnknownSubscriptionID = errors.New("subscriptionID Unknown")
)

var API = resource.APINamespaceRDK.WithComponentType(SubtypeName)

API is a variable that identifies the camera resource API.

Functions

func DecodeImageFromCamera

func DecodeImageFromCamera(ctx context.Context, mimeType string, extra map[string]interface{}, cam Camera) (image.Image, error)

DecodeImageFromCamera retrieves image bytes from a camera resource and serializes it as an image.Image.

func Named

func Named(name string) resource.Name

Named is a helper for getting the named camera's typed resource name.

func NamesFromRobot

func NamesFromRobot(r robot.Robot) []string

NamesFromRobot is a helper for getting all camera names from the given Robot.

func NewPinholeModelWithBrownConradyDistortion

func NewPinholeModelWithBrownConradyDistortion(pinholeCameraIntrinsics *transform.PinholeCameraIntrinsics,
	distortion *transform.BrownConrady,
) transform.PinholeCameraModel

NewPinholeModelWithBrownConradyDistortion creates a transform.PinholeCameraModel from a *transform.PinholeCameraIntrinsics and a *transform.BrownConrady. If *transform.BrownConrady is `nil`, transform.PinholeCameraModel.Distortion is not set & remains nil, to prevent https://go.dev/doc/faq#nil_error.

func NewPropertiesError

func NewPropertiesError(cameraIdentifier string) error

NewPropertiesError returns an error specific to a failure in Properties.

func NewRPCServiceServer

func NewRPCServiceServer(coll resource.APIResourceCollection[Camera]) interface{}

NewRPCServiceServer constructs an camera gRPC service server. It is intentionally untyped to prevent use outside of tests.

func NewUnsupportedImageTypeError

func NewUnsupportedImageTypeError(s ImageType) error

NewUnsupportedImageTypeError is when the stream type is unknown.

func ReadImage

func ReadImage(ctx context.Context, src gostream.VideoSource) (image.Image, func(), error)

ReadImage reads an image from the given source that is immediately available.

Types

type Camera

type Camera interface {
	resource.Resource
	VideoSource
}

A Camera is a resource that can capture frames.

func FromDependencies

func FromDependencies(deps resource.Dependencies, name string) (Camera, error)

FromDependencies is a helper for getting the named camera from a collection of dependencies.

func FromRobot

func FromRobot(r robot.Robot, name string) (Camera, error)

FromRobot is a helper for getting the named Camera from the given Robot.

func FromVideoSource

func FromVideoSource(name resource.Name, src VideoSource, logger logging.Logger) Camera

FromVideoSource creates a Camera resource from a VideoSource. Note: this strips away Reconfiguration and DoCommand abilities. If needed, implement the Camera another way. For example, a webcam implements a Camera manually so that it can atomically reconfigure itself.

func NewClientFromConn

func NewClientFromConn(
	ctx context.Context,
	conn rpc.ClientConn,
	remoteName string,
	name resource.Name,
	logger logging.Logger,
) (Camera, error)

NewClientFromConn constructs a new Client from connection passed in.

type ImageMetadata

type ImageMetadata struct {
	MimeType string
}

ImageMetadata contains useful information about returned image bytes such as its mimetype.

func ReadImageBytes

func ReadImageBytes(ctx context.Context, src gostream.VideoSource, mimeType string) ([]byte, ImageMetadata, error)

ReadImageBytes wraps ReadImage given a mimetype to encode the image as bytes data, returning supplementary metadata for downstream processing. TODO(hexbabe): make function private or remove altogether once the usages are limited to this file.

type ImageType

type ImageType string

ImageType specifies what kind of image stream is coming from the camera.

type ImagesSource

type ImagesSource interface {
	Images(ctx context.Context) ([]NamedImage, resource.ResponseMetadata, error)
}

A ImagesSource is a source that can return a list of images with timestamp.

type NamedImage

type NamedImage struct {
	Image      image.Image
	SourceName string
}

NamedImage is a struct that associates the source from where the image came from to the Image.

type PointCloudSource

type PointCloudSource interface {
	NextPointCloud(ctx context.Context) (pointcloud.PointCloud, error)
}

A PointCloudSource is a source that can generate pointclouds.

type Properties

type Properties struct {
	// SupportsPCD indicates that the Camera supports a valid
	// implementation of NextPointCloud
	SupportsPCD      bool
	ImageType        ImageType
	IntrinsicParams  *transform.PinholeCameraIntrinsics
	DistortionParams transform.Distorter
	MimeTypes        []string
	FrameRate        float32
}

Properties is a lookup for a camera's features and settings.

type VideoSource

type VideoSource interface {
	// Image returns a byte slice representing an image that tries to adhere to the MIME type hint.
	// Image also may return metadata about the frame.
	Image(ctx context.Context, mimeType string, extra map[string]interface{}) ([]byte, ImageMetadata, error)

	// Images is used for getting simultaneous images from different imagers,
	// along with associated metadata (just timestamp for now). It's not for getting a time series of images from the same imager.
	Images(ctx context.Context) ([]NamedImage, resource.ResponseMetadata, error)

	// Stream returns a stream that makes a best effort to return consecutive images
	// that may have a MIME type hint dictated in the context via gostream.WithMIMETypeHint.
	Stream(ctx context.Context, errHandlers ...gostream.ErrorHandler) (gostream.VideoStream, error)

	// NextPointCloud returns the next immediately available point cloud, not necessarily one
	// a part of a sequence. In the future, there could be streaming of point clouds.
	NextPointCloud(ctx context.Context) (pointcloud.PointCloud, error)

	// Properties returns properties that are intrinsic to the particular
	// implementation of a camera.
	Properties(ctx context.Context) (Properties, error)

	// Close shuts down the resource and prevents further use.
	Close(ctx context.Context) error
}

VideoSource represents anything that can capture frames. For more information, see the camera component docs.

Image example:

myCamera, err := camera.FromRobot(machine, "my_camera")
imageBytes, mimeType, err := myCamera.Image(context.Background(), utils.MimeTypeJPEG, nil)

Or try to directly decode as an image.Image:

myCamera, err := camera.FromRobot(machine, "my_camera")
img, err = camera.DecodeImageFromCamera(context.Background(), utils.MimeTypeJPEG, nil, myCamera)

Images example:

myCamera, err := camera.FromRobot(machine, "my_camera")

images, metadata, err := myCamera.Images(context.Background())

Stream example:

myCamera, err := camera.FromRobot(machine, "my_camera")

// gets the stream from a camera
stream, err := myCamera.Stream(context.Background())

// gets an image from the camera stream
img, release, err := stream.Next(context.Background())
defer release()

NextPointCloud example:

myCamera, err := camera.FromRobot(machine, "my_camera")

// gets the next point cloud from a camera
pointCloud, err := myCamera.NextPointCloud(context.Background())

Close example:

myCamera, err := camera.FromRobot(machine, "my_camera")

err = myCamera.Close(context.Background())

func NewVideoSourceFromReader

func NewVideoSourceFromReader(
	ctx context.Context,
	reader gostream.VideoReader,
	syst *transform.PinholeCameraModel, imageType ImageType,
) (VideoSource, error)

NewVideoSourceFromReader creates a VideoSource either with or without a projector. The stream type argument is for detecting whether or not the resulting camera supports return of pointcloud data in the absence of an implemented NextPointCloud function. If this is unknown or not applicable, a value of camera.Unspecified stream can be supplied.

func WrapVideoSourceWithProjector

func WrapVideoSourceWithProjector(
	ctx context.Context,
	source gostream.VideoSource,
	syst *transform.PinholeCameraModel, imageType ImageType,
) (VideoSource, error)

WrapVideoSourceWithProjector creates a Camera either with or without a projector. The stream type argument is for detecting whether or not the resulting camera supports return of pointcloud data in the absence of an implemented NextPointCloud function. If this is unknown or not applicable, a value of camera.Unspecified stream can be supplied.