module

package
v0.51.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 21, 2024 License: AGPL-3.0 Imports: 42 Imported by: 84

Documentation

Overview

Package module provides services for external resource and logic modules.

Module Resource System Overview

The module system allows a user to build an external binary, either in Golang, using this package and any others from the RDK ecosystem, or in any other language, provided it can properly support protobuf/grpc. The path to the binary (the module) and a name for it must be given in the Modules section of the robot config. The normal viam-server (rdk) process will then start this binary, and query it via GRPC for what protocols (protobuf described APIs) and models it supports. Then, any components or services that match will be handled seamlessly by the module, including reconfiguration, shutdown, and dependency management. Modular components may depend on others from either the parent (aka built-in resources) or other modules, and vice versa. Modular resources should behave identically to built-in resources from a user perspective.

Startup

The module manager (modmanager) integrates with the robot and resource manager. During startup, a dedicated GRPC module service is started, listening on a unix socket in a temporary directory (ex: /tmp/viam-modules-893893/parent.sock) and then individual modules are executed. These are each passed dedicated socket address of their own in the same directory, and based on the module name. (ex: /tmp/viam-modules-893893/acme.sock) The parent then queries this address with Ready() and waits for confirmation. The ready response also includes a HandlerMap that defines which protocols and models the module provides support for. The parent then registers these APIs and models, with creator functions that call the manager's AddResource() method. Once all modules are started, normal robot loading continues.

When resources or components are attempting to load that are not built in, their creator method calls AddResource() and a request is built and sent to the module. The entire config is sent as part of this, as are dependencies. Dependencies are passed by name only through GRPC, and the module library on the module side automatically creates grpc clients for each resource, before calling the component/service constructor. In this way, fully usable dependencies are provided, just as they would be during built-in resource creation.

Back on the parent side, once the AddResource() call completes, the modmanager then establishes an rpc client for the resource, and returns that to the resource manager, which inserts it into the resource graph. For built-in protocols (arm, motor, base, etc.) this rpc client is cast to the expected interface, and is functionally identical to a built-in component. For new protocols, the client created is wrapped as a ForeignResource, which (along with the reflection service in the module) allows it to be used normally by external clients that are also aware of the new protocol in question.

Reconfiguration

The reconfiguration process is handled as transparently as possible to the end user. When a resource would be reconfigured by the resource manager, it is checked if it belongs to a module. If true, then a ReconfigureResource() request is sent to the module instead. (The existing grpc client object on the parent side is untouched.) In the module, the receiving method attempts to cast the real resource to registry.ReconfigurableComponent/Service. If successful, the Reconfigure() method is called on the resource. This method receives the full new config (and dependencies) just as AddResource would. It's then up to the resource itself to reconfigure itself accordingly. If the cast fails (e.g. the resource doesn't have the Reconfigure method.) then the existing resource is closed, and a new one created in its place. Note that unlike built-in resources, no proxy resource is used, since the real client is in the parent, and will automatically get the new resource, since it is looked up by name on each function call.

For removal (during shutdown) RemoveResource() is called, and only passes the resource.Name to the module.

Shutdown

The shutdown process is hooked so that during the Close() of the resource manager, resources are checked if they are modular, and if so, RemoveResource() is called after the parent-side rpc client is closed. The grpc module service is also kept open as late as possible. Otherwise, shutdown happens as normal, including the closing of components in topological (dependency) order.

Module Protocol Requirements

A module can technically be built in any language, with or without support of this RDK or other Viam SDKs. From a technical point of view, all that's required is that the module:

  • Is an executable file by unix standards. This can be a compiled binary, or a script with the proper shebang to its interpreter, such as python.
  • Looks at the first argument passed to it at execution, and uses that as it's grpc socket path.
  • Listens with plaintext GRPC on that socket.
  • GRPC must provide the Module service (https://github.com/viamrobotics/api/tree/main/proto/viam/module/v1/module.proto), a reflection service, and any APIs needed for the resources it intends to serve. Note that the "robot" service itself is NOT required.
  • Handles the Module service's calls for Ready(), and Add/Remove/ReconfigureResource()
  • Cleanly exits when sent a SIGINT or SIGTERM signal.

Module Creation Considerations

Under Golang, the module side of things tries to use as much of the "RDK" idioms as possible. Most notably, this includes the registry. So when creating modular components with this package, resources (and protocols) register their "Creator" methods and such during init() or during main(). They then are explicitly added via AddModelFromRegistry() so that merely importing a module doesn't add unneeded/unused grpc services.

In other languages, and for small modules not part of a larger code ecosystem, the registry concept may not make as much sense, and foregoing the registry step in favor of some more direct AddModel() call (which takes the creation handler func directly) may be better.

Index

Constants

View Source 
const (

	// NoModuleParentEnvVar indicates whether there is a parent for a module being started.
	NoModuleParentEnvVar = "VIAM_NO_MODULE_PARENT"
)

Variables

This section is empty.

Functions

func CheckSocketOwner 

func CheckSocketOwner(address string) error

CheckSocketOwner verifies that UID of a filepath/socket matches the current process's UID.

func CreateSocketAddress added in v0.9.0 

func CreateSocketAddress(parentDir, desiredName string) (string, error)

CreateSocketAddress returns a socket address of the form parentDir/desiredName.sock if it is shorter than the socketMaxAddressLength. If this path would be too long, this function truncates desiredName and returns parentDir/truncatedName-hashOfDesiredName.sock.

Importantly, this function will return the same socket address as long as the desiredName doesn't change.

func MakeSelfOwnedFilesFunc added in v0.2.14 

func MakeSelfOwnedFilesFunc(f func() error) error

MakeSelfOwnedFilesFunc calls the given function such that any files made will be self owned.

func ModularMain added in v0.37.0 

func ModularMain(models ...resource.APIModel)

ModularMain can be called as the main function from a module. It will start up a module with all the provided APIModels added to it.

func NewLoggerFromArgs added in v0.2.49 

func NewLoggerFromArgs(moduleName string) logging.Logger

NewLoggerFromArgs can be used to create a logging.Logger at "DebugLevel" if "--log-level=debug" is the third argument in os.Args and at "InfoLevel" otherwise. See config.Module.LogLevel documentation for more info on how to start modules with a "log-level" commandline argument. The created logger will send log events back to the module's parent (the RDK) via gRPC when possible and to STDOUT when not possible.

func NewServer 

func NewServer(opts ...grpc.ServerOption) rpc.Server

NewServer returns a new (module specific) rpc.Server.

Types

type HandlerMap 

type HandlerMap map[resource.RPCAPI][]resource.Model

HandlerMap is the format for api->model pairs that the module will service. Ex: mymap["rdk:component:motor"] = ["acme:marine:thruster", "acme:marine:outboard"].

func NewHandlerMapFromProto 

func NewHandlerMapFromProto(ctx context.Context, pMap *pb.HandlerMap, conn rpc.ClientConn) (HandlerMap, error)

NewHandlerMapFromProto converts protobuf to HandlerMap.

func (HandlerMap) ToProto 

func (h HandlerMap) ToProto() *pb.HandlerMap

ToProto converts the HandlerMap to a protobuf representation.

type Module 

type Module struct {
	pb.UnimplementedModuleServiceServer
	streampb.UnimplementedStreamServiceServer
	robotpb.UnimplementedRobotServiceServer
	// contains filtered or unexported fields
}

Module represents an external resource module that services components/services.

func NewModule 

func NewModule(ctx context.Context, address string, logger logging.Logger) (*Module, error)

NewModule returns the basic module framework/structure. Use ModularMain and NewModuleFromArgs unless you really know what you're doing.

func NewModuleFromArgs 

func NewModuleFromArgs(ctx context.Context) (*Module, error)

NewModuleFromArgs directly parses the command line argument to get its address.

func (*Module) AddModelFromRegistry 

func (m *Module) AddModelFromRegistry(ctx context.Context, api resource.API, model resource.Model) error

AddModelFromRegistry adds a preregistered component or service model to the module's services.

func (*Module) AddResource 

func (m *Module) AddResource(ctx context.Context, req *pb.AddResourceRequest) (*pb.AddResourceResponse, error)

AddResource receives the component/service configuration from the parent.

func (*Module) AddStream added in v0.24.0 

func (m *Module) AddStream(ctx context.Context, req *streampb.AddStreamRequest) (*streampb.AddStreamResponse, error)

AddStream adds a stream. Returns an error if: 1. there is no WebRTC peer connection with viam-sever 2. resource doesn't exist 3. the resource doesn't implement rtppassthrough.Source, 4. there are already the max number of supported tracks on the peer connection 5. SubscribeRTP returns an error 6. A webrtc track is unable to be created 7. Adding the track to the peer connection fails.

func (*Module) Close 

func (m *Module) Close(ctx context.Context)

Close shuts down the module and grpc server.

func (*Module) DiscoverComponents added in v0.45.0 

func (m *Module) DiscoverComponents(
	ctx context.Context,
	req *robotpb.DiscoverComponentsRequest,
) (*robotpb.DiscoverComponentsResponse, error)

DiscoverComponents takes a list of discovery queries and returns corresponding component configurations.

func (*Module) GetParentResource 

func (m *Module) GetParentResource(ctx context.Context, name resource.Name) (resource.Resource, error)

GetParentResource returns a resource from the parent robot by name.

func (*Module) ListStreams added in v0.24.0 

func (m *Module) ListStreams(ctx context.Context, req *streampb.ListStreamsRequest) (*streampb.ListStreamsResponse, error)

ListStreams lists the streams.

func (*Module) OperationManager 

func (m *Module) OperationManager() *operation.Manager

OperationManager returns the operation manager for the module.

func (*Module) PeerConnect added in v0.24.0 

func (m *Module) PeerConnect(encodedOffer string) (string, error)

PeerConnect returns the encoded answer string for the `ReadyResponse`.

func (*Module) Ready 

func (m *Module) Ready(ctx context.Context, req *pb.ReadyRequest) (*pb.ReadyResponse, error)

Ready receives the parent address and reports api/model combos the module is ready to service.

func (*Module) ReconfigureResource 

func (m *Module) ReconfigureResource(ctx context.Context, req *pb.ReconfigureResourceRequest) (*pb.ReconfigureResourceResponse, error)

ReconfigureResource receives the component/service configuration from the parent.

func (*Module) RemoveResource 

func (m *Module) RemoveResource(ctx context.Context, req *pb.RemoveResourceRequest) (*pb.RemoveResourceResponse, error)

RemoveResource receives the request for resource removal.

func (*Module) RemoveStream added in v0.24.0 

func (m *Module) RemoveStream(ctx context.Context, req *streampb.RemoveStreamRequest) (*streampb.RemoveStreamResponse, error)

RemoveStream removes a stream.

func (*Module) SetReady 

func (m *Module) SetReady(ready bool)

SetReady can be set to false if the module is not ready (ex. waiting on hardware).

func (*Module) Start 

func (m *Module) Start(ctx context.Context) error

Start starts the module service and grpc server.

func (*Module) ValidateConfig added in v0.2.25 

func (m *Module) ValidateConfig(ctx context.Context,
	req *pb.ValidateConfigRequest,
) (*pb.ValidateConfigResponse, error)

ValidateConfig receives the validation request for a resource from the parent.

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server provides an rpc.Server wrapper around a grpc.Server.

func (*Server) EnsureAuthed added in v0.11.1 

func (s *Server) EnsureAuthed(ctx context.Context) (context.Context, error)

EnsureAuthed is unsupported.

func (*Server) GRPCHandler 

func (s *Server) GRPCHandler() http.Handler

GRPCHandler is unsupported.

func (*Server) GatewayHandler 

func (s *Server) GatewayHandler() http.Handler

GatewayHandler is unsupported.

func (*Server) InstanceNames 

func (s *Server) InstanceNames() []string

InstanceNames is unsupported.

func (*Server) InternalAddr 

func (s *Server) InternalAddr() net.Addr

InternalAddr returns the internal address of the server.

func (*Server) RegisterServiceServer 

func (s *Server) RegisterServiceServer(
	ctx context.Context,
	svcDesc *grpc.ServiceDesc,
	svcServer interface{},
	svcHandlers ...rpc.RegisterServiceHandlerFromEndpointFunc,
) error

RegisterServiceServer associates a service description with its implementation along with any gateway handlers.

func (*Server) Serve 

func (s *Server) Serve(listener net.Listener) error

Serve begins listening/serving grpc.

func (*Server) ServeHTTP 

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP is unsupported.

func (*Server) ServeTLS 

func (s *Server) ServeTLS(listener net.Listener, certFile, keyFile string, tlsConfig *tls.Config) error

ServeTLS is unsupported.

func (*Server) Start 

func (s *Server) Start() error

Start is unsupported.

func (*Server) Stop 

func (s *Server) Stop() error

Stop performs a GracefulStop() on the underlying grpc server.

Source Files

View all Source files

Directories

Path Synopsis
Package modmanager provides the module manager for a robot.
 Package modmanager provides the module manager for a robot.
options
Package modmanageroptions provides Options for configuring a mod manager
 Package modmanageroptions provides Options for configuring a mod manager
Package modmaninterface abstracts the manager interface to avoid an import cycle/loop.
 Package modmaninterface abstracts the manager interface to avoid an import cycle/loop.
Package main is a module designed to help build tests for reconfiguration logic between module versions
 Package main is a module designed to help build tests for reconfiguration logic between module versions
Package main is a module for testing, with an inline generic component to return internal data and perform other test functions.
 Package main is a module for testing, with an inline generic component to return internal data and perform other test functions.
Package main is a module for testing, with an inline generic component to return internal data and perform other test functions.
 Package main is a module for testing, with an inline generic component to return internal data and perform other test functions.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.