agent

Documentation

Index

• type Action
• type AgentInterface
• type AgentProxyWithWorldCache
  • func (agent *AgentProxyWithWorldCache) GetServerFunc(loops *looper.Manager) func()
  • func (agent *AgentProxyWithWorldCache) Init(actionSpace map[string]SpaceSpec, observationSpace map[string]SpaceSpec) map[string]string
  • func (world *AgentProxyWithWorldCache) InitWorld(details map[string]string) (map[string]SpaceSpec, map[string]SpaceSpec)
  • func (world *AgentProxyWithWorldCache) Observe(name string) etensor.Tensor
  • func (world *AgentProxyWithWorldCache) ObserveWithShape(name string, shape SpaceSpec) etensor.Tensor
  • func (agent *AgentProxyWithWorldCache) StartServer()
  • func (agent *AgentProxyWithWorldCache) Step(observations map[string]etensor.Tensor, debug string) map[string]Action
  • func (world *AgentProxyWithWorldCache) StepWorld(actions map[string]Action, agentDone bool) (bool, string)
• type SpaceSpec
• type WorldInterface

Types

type Action

type Action struct {
	ActionShape    *SpaceSpec     `desc:"Optional description of the action."`
	Vector         etensor.Tensor `desc:"A vector describing the action. For example, this might be joint positions or forces applied to actuators."`
	DiscreteOption int            `desc:"Choice from among the DiscreteLabels in Continuous."`
}

An Action describes what the agent is doing at a given timestep. It should contain either a continuous vector or a discrete option, as specified by its shape.

type AgentInterface

type AgentInterface interface {
	// Init passes variables to the Agent: Action space, Observation space, and initial Observation. It receives any specification in the form of a string which the agent chooses to provide. Agent should reinitialize the network for the beginning of a new run.
	// details lets the agent request that the world be configured to spec.
	Init(actionSpace map[string]SpaceSpec, observationSpace map[string]SpaceSpec) (details map[string]string)

	// Step takes in a map of named Observations. It returns a map of named Actions. The observation can be expected to conform to the shape given in Init, and the Action should conform to the action specification given there. The debug string is for debug information from the environment and should not be used for real training or evaluation.
	Step(observations map[string]etensor.Tensor, debug string) (actions map[string]Action)
}

AgentInterface allows the Agent to provide actions given observations. This allows the agent to be embedded within a world.

type AgentProxyWithWorldCache

type AgentProxyWithWorldCache struct {
	AgentInterface
	WorldInterface

	CachedObservations map[string]etensor.Tensor `desc:"Observations from the last step."`
	CachedActions      map[string]Action         `desc:"Actions the action wants to take this step."`
	// contains filtered or unexported fields
}

AgentProxyWithWorldCache represents the agent using the AgentInterface, so that the agent can be called with a Step function. At the same time, it implements the WorldInterface, allowing this same object to represent the world internally to the agent. It holds caches for observations and actions, acting like a mailbox between where these observations and actions can be stashed.

func (*AgentProxyWithWorldCache) GetServerFunc

func (agent *AgentProxyWithWorldCache) GetServerFunc(loops *looper.Manager) func()

GetServerFunc returns a function that can be used to start the server.

func (*AgentProxyWithWorldCache) Init

func (agent *AgentProxyWithWorldCache) Init(actionSpace map[string]SpaceSpec, observationSpace map[string]SpaceSpec) map[string]string

Init the agent. This tells the agent what shape input/output to use, and gets some suggestions from the agent about what the world should be like.

func (*AgentProxyWithWorldCache) InitWorld

func (world *AgentProxyWithWorldCache) InitWorld(details map[string]string) (map[string]SpaceSpec, map[string]SpaceSpec)

Init sets up a server and waits for the agent to handshake with it for initiation.

func (*AgentProxyWithWorldCache) Observe

func (world *AgentProxyWithWorldCache) Observe(name string) etensor.Tensor

Observe returns an observation from the cache.

func (*AgentProxyWithWorldCache) ObserveWithShape

func (world *AgentProxyWithWorldCache) ObserveWithShape(name string, shape SpaceSpec) etensor.Tensor

ObserveWithShape Returns a tensor for the named modality like Observe. This allows the agent to request an observation in a specific shape, which may involve downsampling. It should throw an error if the shap can't be satisfied.

func (*AgentProxyWithWorldCache) StartServer

func (agent *AgentProxyWithWorldCache) StartServer()

StartServer blocks, and sometimes calls Init or Step.

func (*AgentProxyWithWorldCache) Step

func (agent *AgentProxyWithWorldCache) Step(observations map[string]etensor.Tensor, debug string) map[string]Action

Step the agent. Internally, this calls looper.Manager.Step. It provides observations to the agent, and records what actions were taken, using the caches on the WorldInterface to move them in and out.

func (*AgentProxyWithWorldCache) StepWorld

func (world *AgentProxyWithWorldCache) StepWorld(actions map[string]Action, agentDone bool) (bool, string)

StepWorld steps the world one step. This function is called internally by the agent as a way of recording what actions it is taking that step.

type SpaceSpec

type SpaceSpec struct {
	// Continuous
	ContinuousShape []int   `desc:"The dimensions of an array. For example, [3,2] would be a 3 by 2 array. [1] would be a single value."`
	Stride          []int   `desc:"TODO Replace Shape and Stride with one Shape object, like an etensor.Tensor has."`
	Min             float64 `desc:"The minimum continuous value."`
	Max             float64 `desc:"The maximum continuous value."`

	// Discrete
	DiscreteLabels []string `` /* 163-byte string literal not displayed */
}

The SpaceSpec of an action or observation. It should *either* be continuous with a SpaceSpec or discrete, but not both. If both are set, treat it as continuous, using SpaceSpec.

type WorldInterface

type WorldInterface interface {
	// InitWorld Initializes or reinitialize the world. This blocks until it hears from the world that it has been initialized. It returns the specifications for the action and observation spaces as its two return arguments.
	// The details allow the Agent to request that the world configure itself in some way.
	// The Action Space describes the shape and names of what the model can send as outputs. This will be constant across the run.
	// The Observation Space describes the shape and names of what the model can expect as inputs. This will be constant across the run.
	// InitWorld and StepWorld are so named to allow a single object to implement both this interface and the AgentInterface.
	InitWorld(details map[string]string) (actionSpace map[string]SpaceSpec, observationSpace map[string]SpaceSpec)

	// StepWorld the environment. It takes in a set of actions and returns observations and a debug string. // TODO Recomment
	// The actions should conform to the action space specification.
	// The observations can be expected to conform to the observation space specification. The observations will be cached such that a separate function can get them before the next time Step is called.
	// The debug string should not be used for actual training.
	StepWorld(actions map[string]Action, agentDone bool) (done bool, debug string)

	// Observe returns a cached tensor for the named modality. E.g. “x” or “vision” or “reward”. This just returns a cached entry into the map gotten the last time Step was called.
	Observe(name string) etensor.Tensor
}

WorldInterface is a minimal interface for the world. It allows the agent to initialize it, step it with actions, and get observations from it.