muniverse

README

μniverse

μniverse is my attempt to wrap a bunch of HTML5 games in a single API. This is similar to openai/universe, except that it avoids Flash and VNC. For more on this, see Advantages over Universe

This is a work in progress. See Roadmap for more details.

Getting Started

µniverse depends on a few software projects. It requires fairly new versions, so package managers like apt-get may not be suitable. Here are the packages and links to downloads:

• Docker >= 17.03
• Go >= 1.8

Installing Go is particularly involved, as you will want to create a GOPATH. See here for details on that.

Once you have Go and Docker, you can download µniverse in one easy step:

go get github.com/unixpickle/muniverse

The first time you use µniverse, it will pull a somewhat large Docker container. I recommend you do this manually before you use µniverse:

docker pull unixpickle/muniverse:0.115.0

Once you have µniverse installed, you may want to install bindings for other languages such as Python. If you want to use µniverse from Go, muniverse-agent might serve as a good starting point. You may also want to checkout the GoDoc for API details.

Advantages over Universe

Compared to OpenAI Universe, μniverse will give the following advantages:

• No need to play games in real-time.
  • No "falling behind" on slower computers
  • Play games faster than real-time on fast machines.
• No need for a neural network to read scores from screenshots.
• Fewer glitches due to menu automation.
• No unsafe actions (e.g. pressing the Main Menu button).
• No Docker containers without accompanying source code
  • All code to generate containers is included
  • Open source scripts to download & package games

Most of the above advantages come from focusing on HTML5 games rather than Flash games.

Contents

• this directory - high-level Go API for controlling environments
• chrome/ - Go API for controlling a headless Chrome instance
• games/ - scripts for downloading & packaging games
• container/ - build files for the Docker container
• codegen/ - small program to auto-generate Go games registry
• util/ - small tools which come in handy while using µniverse.
• bindings/ - bindings for other programming languages.

Roadmap

Here's what is done so far:

• Go API for interfacing with headless Chrome.
• Mechanism for downloading & packaging games.
• JavaScript interface for controlling time (timers, Date, etc.)
• Docker container for running headless Chrome.
• Go API for controlling Docker containers.
• Simple interface for gym-like environment control.
• Python bindings

Here's a (non-exhaustive) to-do list:

• Get more games.
• Gym integration.
• Get WebGL to work without occasional failures.
• Better way to verify connection to keep-alive socket.
• Rewrite download script in Go:
  • Faster (jq is slow)
  • Support absolute paths? (how to specify destination...)
• Cleaner Windows IP address lookups

License

This is under a BSD 2-clause license. See LICENSE.

Documentation

Index

• Variables
• func ObsPNG(obs Obs) ([]byte, error)
• func RGB(o Obs) (buffer []uint8, width, height int, err error)
• type Env
  • func CursorEnv(e Env, initX, initY int) Env
  • func NewEnv(spec *EnvSpec) (Env, error)
  • func NewEnvOptions(spec *EnvSpec, opts *Options) (e Env, err error)
  • func RecordEnv(e Env, dir string) Env
• type EnvSpec
  • func SpecForName(name string) *EnvSpec
• type Obs
• type Options
• type Recording
  • func CreateRecording(path string) (rec *Recording, err error)
  • func OpenRecording(path string) (rec *Recording, err error)
  • func (r *Recording) NumObs() int
  • func (r *Recording) NumSteps() int
  • func (r *Recording) ReadObs(idx int) (obs Obs, err error)
  • func (r *Recording) ReadStep(idx int) (info *StepInfo, err error)
  • func (r *Recording) WriteObs(obs Obs) (err error)
  • func (r *Recording) WriteStep(info *StepInfo) (err error)
• type StepInfo

Variables

var EnvSpecs = []*EnvSpec{}/* 202 elements not displayed */

Functions

func ObsPNG

func ObsPNG(obs Obs) ([]byte, error)

ObsPNG encodes an observation as PNG data.

func RGB

func RGB(o Obs) (buffer []uint8, width, height int, err error)

RGB generates an RGB pixel buffer for an observation.

The buffer is packed the same way as the pixel data in *image.RGBA, but without an alpha channel.

Types

type Env

type Env interface {
	// Spec returns details about the environment.
	Spec() *EnvSpec

	// Reset resets the environment to a start state.
	Reset() error

	// Step sends the given events and advances the
	// episode by the given amount of time.
	//
	// If done is true, then the episode has ended.
	// After an episode ends, Reset must be called once
	// before Step may be called again.
	// However, observations may be made even after the
	// episode has ended.
	//
	// Typical event types are *chrome.MouseEvent and
	// *chrome.KeyEvent.
	Step(t time.Duration, events ...interface{}) (reward float64,
		done bool, err error)

	// Observe produces an observation for the current
	// state of the environment.
	Observe() (Obs, error)

	// Close cleans up resources used by the environment.
	//
	// After Close is called, the Env should not be used
	// anymore by any Goroutine.
	Close() error

	// Log returns internal log messages.
	// For example, it might return information about 404
	// errors.
	//
	// The returned list is a copy and may be modified by
	// the caller.
	Log() []string
}

An Env controls and observes an environment.

It is not safe to run an methods on an Env from more than one Goroutine at a time.

The lifecycle of an environment is as follows: First, Reset is called to start an episode. Then, Step and Observe may be called repeatedly in any order until Step returns done=true to signal that the episode has ended. Once the episode has ended, Observe may be called but Step may not be. Call Reset to start a new episode and begin the process over again.

When you are done with an Env, you must close it to clean up resources associated with it.

func CursorEnv

func CursorEnv(e Env, initX, initY int) Env

CursorEnv creates a wrapped environment which renders a cursor at the current mouse location.

At every episode, the mouse is initialized to the given x and y coordinates.

By default, Chrome will not render a mouse. Thus, it is necessary to render a mouse manually.

When the resulting Env is closed, e is closed as well.

func NewEnv

func NewEnv(spec *EnvSpec) (Env, error)

NewEnv creates a new environment inside the default Docker image with the default settings.

This may take a few minutes to run the first time, since it has to download a large Docker image. If the download takes too long, NewEnv may time out. If this happens, it is recommended that you download the image manually.

func NewEnvOptions

func NewEnvOptions(spec *EnvSpec, opts *Options) (e Env, err error)

NewEnvOptions creates a new environment with the given set of options.

func RecordEnv

func RecordEnv(e Env, dir string) Env

RecordEnv creates an Env wrapper which stores a new Recording for every episode.

Recordings are stored inside the given directory. Each recording is assigned a pseudo-random directory name to prevent collisions. However, the names all begin with "recording_".

Closing the resulting Env will automatically close e.

type EnvSpec

type EnvSpec struct {
	Name      string `bson:"Name"`
	BaseURL   string `bson:"BaseURL"`
	Width     int    `bson:"Width"`
	Height    int    `bson:"Height"`
	AllCanvas bool   `bson:"AllCanvas"`
	Options   string `bson:"Options"`

	KeyWhitelist  []string `bson:"KeyWhitelist,omitempty"`
	MouseType     string   `bson:"MouseType"`
	MouseRequired bool     `bson:"MouseRequired"`

	VariantOf string `bson:"VariantOf"`
}

An EnvSpec contains meta-data about an environment.

func SpecForName

func SpecForName(name string) *EnvSpec

SpecForName finds the first entry in EnvSpecs with the given name. If no entry is found, nil is returned.

type Obs

type Obs interface {
	Image() (image.Image, error)
}

Obs is an observation from an environment.

type Options

type Options struct {
	// CustomImage, if non-empty, specifies a custom
	// Docker image to use for the environment.
	CustomImage string

	// GamesDir, if non-empty, specifies a directory on
	// the host to mount into the Docker container as a
	// downloaded_games folder.
	GamesDir string

	// DevtoolsHost, if non-empty, specifies the host of
	// an already-running Chrome's DevTools server.
	//
	// If this is non-empty, a Docker container will not
	// be launched, and the CustomImage and GamesDir
	// options should not be used.
	// Further, the GameHost field must be set.
	DevtoolsHost string

	// GameHost, if non-empty, specifies the host for the
	// downloaded_games file server.
	// This can only be used in conjunction with
	// DevtoolsHost.
	GameHost string

	// Compression, if set, allows observations to be
	// compressed in a lossy fashion.
	// Using compression may provide a performance boost, but
	// at the expense of observation integrity.
	Compression bool

	// CompressionQuality controls the quality of compressed
	// observations if Compression is set.
	// The value ranges from 0 to 100 (inclusive).
	CompressionQuality int
}

Options specifies how to configure a new Env.

By default, environments run inside of Docker. In this case, the Docker image can be manually specified with CustomImage, and/or a custom games directory can be specified with GamesDir.

Sometimes, it might be desirable to run an Env directly inside an arbitrary instance of Chrome without starting a Docker container. In this case, you can set the DevtoolsHost field. You will also need to manually start a file-system server for your downloaded_games directory and point GameHost to this server.

type Recording

type Recording struct {
	// contains filtered or unexported fields
}

Recording is an on-disk record of observations and actions in an environment rollout.

It is not safe to run methods on a Recording from more than one Goroutine at a time. Further, it is not safe to open a Recording if another Goroutine or program is writing to it.

func CreateRecording

func CreateRecording(path string) (rec *Recording, err error)

CreateRecording creates a new Recording at the given path.

func OpenRecording

func OpenRecording(path string) (rec *Recording, err error)

OpenRecording opens an existing Recording from a directory at the given path.

func (*Recording) NumObs

func (r *Recording) NumObs() int

NumObs returns the number of recorded observations.

func (*Recording) NumSteps

func (r *Recording) NumSteps() int

NumSteps returns the number of recorded steps.

func (*Recording) ReadObs

func (r *Recording) ReadObs(idx int) (obs Obs, err error)

ReadObs reads the observation at the given index. It fails if the index is out of range.

func (*Recording) ReadStep

func (r *Recording) ReadStep(idx int) (info *StepInfo, err error)

ReadStep reads the step at the given index. It fails if the index is out of range.

func (*Recording) WriteObs

func (r *Recording) WriteObs(obs Obs) (err error)

WriteObs adds an observation to the Recording.

func (*Recording) WriteStep

func (r *Recording) WriteStep(info *StepInfo) (err error)

WriteStep adds a step to the Recording.

type StepInfo

type StepInfo struct {
	Time   time.Duration
	Events []interface{}

	Reward float64
	Done   bool
}

StepInfo stores information about a step in an environment and the result of that step.