README
¶
Env
Env provides the environment interfaces for the agents. It contains a means of spinning up local Sphere servers and an extra wrapper around the sphere gRPC interface.
The package also includes normalization tools for transorming incoming state in norm.go. For instance
one could expand the dimensions of a tensor or discretize continuous state.
Usage
Create a local Gym server in a docker container and connect to it.
server, _ := NewLocalServer(GymServerConfig)
defer server.Close()
Find a local Gym server or create one.
server, _ := FindOrCreate(GymServerConfig)
defer server.Close()
Create an environment
env, _ := server.Make("CartPole-v0")
Create an environment with a normalizer which will expand the dimensions along the 0 axis.
env, _ := server.Make("CartPole-v0", WithNormalizer(envv1.NewExpandDimsNormalizer(0)))
Create an environment and start recording video and statistics.
env, _ := server.Make("CartPole-v0", WithRecorder())
Take a step in the env.
outcome, _ := env.Step(action)
Reset the env.
init, _ := env.Reset()
End the env, printing results if present and then deleting any videos and statistics.
env.End()
Documentation
¶
Overview ¶
Package env provides a wrapper for interacting with Sphere as well as normalization tools.
Index ¶
- Variables
- func PotentialsShape(space *spherev1alpha.Space) []int
- func SpaceMinMax(space *sphere.Space) (min, max *tensor.Dense, err error)
- func SpaceShape(space *spherev1alpha.Space) []int
- func WithDialOpts(dialOpts ...grpc.DialOption) func(*Server)
- func WithGoalNormalizer(normalizer Normalizer) func(*Env)
- func WithLogger(logger *log.Logger) func(*Env)
- func WithNormalizer(normalizer Normalizer) func(*Env)
- func WithRecorder() func(*Env)
- func WithServerLogger(logger *log.Logger) func(*Server)
- func WithWrapper(wrapper spherev1alpha.IsEnvWrapper) func(*Env)
- type BoxSpace
- type Env
- func (e *Env) ActionSpaceShape() []int
- func (e *Env) BoxSpace() (*BoxSpace, error)
- func (e *Env) Clean()
- func (e *Env) Close() error
- func (e *Env) End()
- func (e *Env) MaxSteps() int
- func (e *Env) ObservationSpaceShape() []int
- func (e *Env) PlayAll()
- func (e *Env) Print()
- func (e *Env) PrintResults() error
- func (e *Env) Render() (*spherev1alpha.Image, error)
- func (e *Env) Reset() (init *InitialState, err error)
- func (e *Env) Results() (*Results, error)
- func (e *Env) SampleAction() (int, error)
- func (e *Env) Step(value int) (*Outcome, error)
- func (e *Env) Videos(path string) ([]string, error)
- type EqWidthBinNormalizer
- type ExpandDimsNormalizer
- type InitialState
- type LocalServerConfig
- type MinMaxNormalizer
- type Normalizer
- type Opt
- type Outcome
- type ReshapeNormalizer
- type Results
- type Server
- type ServerOpts
Constants ¶
This section is empty.
Variables ¶
var DefaultAtariWrapper = &spherev1alpha.EnvWrapper_DeepmindAtariWrapper{ DeepmindAtariWrapper: &spherev1alpha.DeepmindAtariWrapper{EpisodeLife: true, ClipRewards: true}, }
DefaultAtariWrapper is the default deepmind atari wrapper.
var GymServerConfig = &LocalServerConfig{Image: "sphereproject/gym", Version: "latest", Port: "50051/tcp"}
GymServerConfig is a configuration for a OpenAI Gym server environment.
Functions ¶
func PotentialsShape ¶
func PotentialsShape(space *spherev1alpha.Space) []int
PotentialsShape is an overloaded method that will return a dense tensor of potentials for a given space.
func SpaceMinMax ¶
SpaceMinMax returns the min/max for a space as tensors. Sphere already normalizes infinite spaces to floats.
func SpaceShape ¶
func SpaceShape(space *spherev1alpha.Space) []int
SpaceShape return the shape of the given space.
func WithDialOpts ¶
func WithDialOpts(dialOpts ...grpc.DialOption) func(*Server)
WithDialOpts adds dial options to the server connection.
func WithGoalNormalizer ¶
func WithGoalNormalizer(normalizer Normalizer) func(*Env)
WithGoalNormalizer adds a normalizer for goal data.
func WithLogger ¶
WithLogger adds a logger to the env.
func WithNormalizer ¶
func WithNormalizer(normalizer Normalizer) func(*Env)
WithNormalizer adds a normalizer for observation data.
func WithServerLogger ¶
WithServerLogger adds a logger to the server.
func WithWrapper ¶
func WithWrapper(wrapper spherev1alpha.IsEnvWrapper) func(*Env)
WithWrapper adds an environment wrapper.
Types ¶
type BoxSpace ¶
type BoxSpace struct {
// High values for this space.
High *tensor.Dense
// Low values for this space.
Low *tensor.Dense
// Shape of the space.
Shape []int
}
BoxSpace is a helper for box spaces that converts the values to dense tensors. TODO: make proto plugin to do this automagically (protoc-gen-tensor)
type Env ¶
type Env struct {
*spherev1alpha.Environment
// Client to connect to the Sphere server.
Client spherev1alpha.EnvironmentAPIClient
// VideoPaths of result videos downloadloaded from the server.
VideoPaths []string
// Normalizer normalizes observation data.
Normalizer Normalizer
// GoalNormalizer normalizes goal data.
GoalNormalizer Normalizer
// contains filtered or unexported fields
}
Env is a convienience environment wrapper.
func (*Env) ActionSpaceShape ¶
ActionSpaceShape is the shape of the action space. TODO: should this be in the API of off the generated code?
func (*Env) End ¶
func (e *Env) End()
End is a helper function that will close an environment and return the results and play any videos.
func (*Env) ObservationSpaceShape ¶
ObservationSpaceShape is the shape of the observation space.
func (*Env) PrintResults ¶
PrintResults results for the environment.
func (*Env) Render ¶
func (e *Env) Render() (*spherev1alpha.Image, error)
Render the environment. TODO: should maybe be a stream.
func (*Env) SampleAction ¶
SampleAction returns a sample action for the environment.
type EqWidthBinNormalizer ¶
type EqWidthBinNormalizer struct {
// contains filtered or unexported fields
}
EqWidthBinNormalizer is an EqWidthBinner applied using tensors.
func NewEqWidthBinNormalizer ¶
func NewEqWidthBinNormalizer(intervals *tensor.Dense) *EqWidthBinNormalizer
NewEqWidthBinNormalizer is a new dense binner.
func (*EqWidthBinNormalizer) Init ¶
func (d *EqWidthBinNormalizer) Init(e *Env) error
Init the normalizer.
type ExpandDimsNormalizer ¶
type ExpandDimsNormalizer struct {
// contains filtered or unexported fields
}
ExpandDimsNormalizer will expand the dims of the state.
func NewExpandDimsNormalizer ¶
func NewExpandDimsNormalizer(axis int) *ExpandDimsNormalizer
NewExpandDimsNormalizer returns a new expand dims normalizer.
func (*ExpandDimsNormalizer) Init ¶
func (r *ExpandDimsNormalizer) Init(e *Env) error
Init the normalizer.
type InitialState ¶
type InitialState struct {
// Observation of the environment.
Observation *tensor.Dense
// Goal if present.
Goal *tensor.Dense
}
InitialState of the environment.
type LocalServerConfig ¶
type LocalServerConfig struct {
// Docker image of environment.
Image string
// Version of the docker image.
Version string
// Port the environment is exposed on.
Port string
// Logger for the server.
Logger *log.Logger
}
LocalServerConfig is the environment server config.
type MinMaxNormalizer ¶
type MinMaxNormalizer struct {
// contains filtered or unexported fields
}
MinMaxNormalizer is a min/max normalizer that makes all values between 0>x<1.
func NewMinMaxNormalizer ¶
func NewMinMaxNormalizer() *MinMaxNormalizer
NewMinMaxNormalizer returns a new min/max
func (*MinMaxNormalizer) Init ¶
func (m *MinMaxNormalizer) Init(e *Env) (err error)
Init the normalizer.
type Normalizer ¶
type Normalizer interface {
// Init the normalizer.
Init(env *Env) error
// Norm normalizes the input data.
Norm(input *tensor.Dense) (*tensor.Dense, error)
}
Normalizer will normalize the data coming from an environment.
type Outcome ¶
type Outcome struct {
// Observation of the current state.
Observation *tensor.Dense
// Action that was taken
Action int
// Reward from action.
Reward float32
// Whether the environment is done.
Done bool
}
Outcome of taking an action.
type ReshapeNormalizer ¶
type ReshapeNormalizer struct {
// contains filtered or unexported fields
}
ReshapeNormalizer will reshape the state.
func NewReshapeNormalizer ¶
func NewReshapeNormalizer(shape tensor.Shape) *ReshapeNormalizer
NewReshapeNormalizer returns a new reshape normalizer.
func (*ReshapeNormalizer) Init ¶
func (r *ReshapeNormalizer) Init(e *Env) error
Init the normalizer.
type Results ¶
type Results struct {
// Episodes is a map of episode id to result.
Episodes map[int32]*spherev1alpha.EpisodeResult
// Videos is a map of episode id to result.
Videos map[int32]*spherev1alpha.Video
// AverageReward is the average reward of the episodes.
AverageReward float32
}
Results from an environment run.
type Server ¶
type Server struct {
// Client to connect to the Sphere server.
Client spherev1alpha.EnvironmentAPIClient
// contains filtered or unexported fields
}
Server of environments.
func FindOrCreate ¶
func FindOrCreate(config *LocalServerConfig) (*Server, error)
FindOrCreate will find a local server for the given config or create one.
func NewLocalServer ¶
func NewLocalServer(config *LocalServerConfig) (*Server, error)
NewLocalServer creates a new environment server by launching a docker container and connecting to it.
Source Files