Documentation
¶
Overview ¶
Package env defines an interface for environments, which determine the nature and sequence of States that can be used as inputs to a model and it can also accept Action responses from the model that affect how the enviroment evolves in the future.
By adhering to this interface, it is then easier to mix-and-match environments with models.
The overall division of labor is that the model keeps track of the outer-most Run time-scale depending on its own parameters and learning trajectory and the environment is responsible for generating patterns for each run.
Multiple different environments will typically be used in a model, e.g., one for training and other(s) for testing. Even if these envs all share a common database of patterns, a different Env should be used for each case where different counters and sequences of events etc are presented, which keeps them from interfering with each other. Also, the etable.IdxView can be used to allow multiple different Env's to all present different indexed views into a shared common etable.Table (e.g., train / test splits). The basic FixedTable env implementation uses this.
Thus, the Env encapsulates all of the counter management logic for each aspect of model training and testing, so that the model itself just needs to manange which Env to use, when, and manage the connection of the Env States as inputs to the model, and vice-versa for Actions on the Env coming from the model.
The channel allows annotation about multiple possible channels of I/O and any given I/O event can specify which channels are provided. Particular paradigms of environments must establish naming conventions for these channels which then allow the model to use the information appropriately -- the Env interface only provides the most basic framework for establishing these paradigms, and ultimately a given model will only work within a particular paradigm of environments following specific conventions.
See e.g., env.FixedTable for particular implementation of a fixed Table of patterns, for one example of a widely-used paradigm.
Typically each specific implementation of this Env interface will have multiple parameters etc that can be modified to control env behavior -- all of this is paradigm-specific and outside the scope of this basic interface.
See the emergent github wiki for more info: https://github.com/emer/emergent/wiki/Env
Index ¶
- Variables
- func CounterChg(en Env, scale TimeScales) bool
- func CounterCur(en Env, scale TimeScales) int
- func CounterPrv(en Env, scale TimeScales) int
- func SchemaFromScales(ts []TimeScales) etable.Schema
- type Ctr
- type Element
- type Elements
- type Env
- type FixedTable
- func (ft *FixedTable) Action(element string, input etensor.Tensor)
- func (ft *FixedTable) Actions() Elements
- func (ft *FixedTable) Counter(scale TimeScales) (cur, prv int, chg bool)
- func (ft *FixedTable) Counters() []TimeScales
- func (ft *FixedTable) Desc() string
- func (ft *FixedTable) Init(run int)
- func (ft *FixedTable) Name() string
- func (ft *FixedTable) Row() int
- func (ft *FixedTable) SetTrialName()
- func (ft *FixedTable) State(element string) etensor.Tensor
- func (ft *FixedTable) States() Elements
- func (ft *FixedTable) Step() bool
- func (ft *FixedTable) Validate() error
- type TimeScales
Constants ¶
This section is empty.
Variables ¶
var KiT_TimeScales = kit.Enums.AddEnum(TimeScalesN, false, nil)
Functions ¶
func CounterChg ¶
func CounterChg(en Env, scale TimeScales) bool
CounterChg returns whether counter changed during last Step() this Counter for Python because it cannot process multiple return values
func CounterCur ¶
func CounterCur(en Env, scale TimeScales) int
CounterCur returns current counter state for given time scale this Counter for Python because it cannot process multiple return values
func CounterPrv ¶
func CounterPrv(en Env, scale TimeScales) int
CounterPrv returns previous counter state for given time scale this Counter for Python because it cannot process multiple return values
func SchemaFromScales ¶
func SchemaFromScales(ts []TimeScales) etable.Schema
SchemaFromScales returns an etable.Schema suitable for creating an etable.Table to record the given list of time scales. Can then add to this schema anything else that might be needed, before using it to create a Table.
Types ¶
type Ctr ¶
type Ctr struct {
Scale TimeScales `inactive:"+" desc:"the unit of time scale represented by this counter (just FYI)"`
Cur int `desc:"current counter value"`
Prv int `view:"-" desc:"previous counter value, prior to last Incr() call (init to -1)"`
Chg bool `view:"-" desc:"did this change on the last Step() call or not?"`
Max int `` /* 126-byte string literal not displayed */
}
Ctr is a counter that counts increments at a given time scale. It keeps track of when it has been incremented or not, and retains the previous value.
func (*Ctr) Incr ¶
Incr increments the counter by 1. If Max > 0 then if Incr >= Max the counter is reset to 0 and true is returned. Otherwise false.
type Element ¶
type Element struct {
Name string `desc:"name of this element -- must be unique"`
Shape []int `` /* 160-byte string literal not displayed */
DimNames []string `desc:"names of the dimensions within the Shape -- optional but useful for ensuring correct usage"`
}
Element specifies one element of State or Action in an environment
func (*Element) FromColumn ¶
FromColumn copies element data from etable Column that describes an etable.Table
type Elements ¶
type Elements []Element
Elements is a list of Element info
func (*Elements) FromSchema ¶
FromSchema copies element data from a etable Schema that describes an etable.Table
type Env ¶
type Env interface {
// Name returns a name for this environment, which can be useful
// for selecting from a list of options etc.
Name() string
// Desc returns an (optional) brief description of this particular
// environment
Desc() string
// Validate checks if the various specific parameters for this
// Env have been properly set -- if not, error message(s) will
// be returned. If everything is OK, nil is returned, in which
// case calls to Counters(), States(), and Actions() should all
// return valid data. It is essential that a model *always* check
// this as a first step, because the Env will not generally check
// for errors on any subsequent calls (for greater efficiency
// and simplicity) and this call can also establish certain general
// initialization settings that are not run-specific and thus make
// sense to do once at this point, not every time during Init().
Validate() error
// Counters returns []TimeScales list of counters supported by this env.
// These should be consistent within a paradigm and most models
// will just expect particular sets of counters, but this can be
// useful for sanity checking that a suitable env has been selected.
// See SchemaFromScales function that takes this list of time
// scales and returns an etable.Schema for Table columns to record
// these counters in a log.
Counters() []TimeScales
// States returns a list of Elements of tensor outputs that this env
// generates, specifying the unique Name and Shape of the data.
// This information can be derived directly from an etable.Schema
// and used for configuring model input / output pathways to fit
// with those provided by the environment. Depending on the
// env paradigm, all elements may not be always available at every
// point in time e.g., an env might alternate between Action and Reward
// elements. This may return nil if Env has not been properly
// configured.
States() Elements
// Actions returns a list of elements of tensor inputs that this env
// accepts, specifying the unique Name and Shape of the data.
// Specific paradigms of envs can establish the timing and function
// of these inputs, and how they then affect subsequent outputs
// e.g., if the model is required to make a particular choice
// response and then it can receive a reward or not contingent
// on that choice.
Actions() Elements
// Init initializes the environment for a given run of the model.
// The environment may not care about the run number, but may implement
// different parameterizations for different runs (e.g., between-subject
// manipulations). In general the Env can expect that the model will likely
// have established a different random seed per run, prior to calling this
// method, and that may be sufficient to enable different run-level behavior.
// All other initialization / updating beyond this outer-most Run level must
// be managed internally by the Env itself, and the model can query the
// Counter state information to determine when things have updated at different
// time scales. See Step() for important info about state of env after Init
// but prior to first Step() call.
Init(run int)
// Step generates the next step of environment state.
// This is the main API for how the model interacts with the environment --
// the env should update all other levels of state internally over
// repeated calls to the Step method.
// If there are no further inputs available, it returns false (most envs
// typically only return true and just continue running as long as needed).
//
// The Env thus always reflects the *current* state of things, and this
// call increments that current state, such that subsequent calls to
// State(), Counter() etc will return this current state.
// This implies that the state just after Init and prior to first Step
// call should be an *initialized* state that then allows the first Step
// call to establish the proper *first* state. Typically this means that
// one or more counters will be set to -1 during Init and then get incremented
// to 0 on the first Step call.
Step() bool
// State returns the given element's worth of tensor data from the environment
// based on the current state of the env, as a function of having called Step().
// If no output is available on that element, then nil is returned.
// The returned tensor must be treated as read-only as it likely points to original
// source data -- please make a copy before modifying (e.g., Clone() methdod)
State(element string) etensor.Tensor
// Action sends tensor data about e.g., responses from model back to act
// on the environment and influence its subsequent evolution.
// The nature and timing of this input is paradigm dependent.
Action(element string, input etensor.Tensor)
// Counter(scale TimeScales) returns current counter state for given time scale,
// the immediate previous counter state, and whether that time scale changed
// during the last Step() function call (this may be true even if cur == prv, if
// the Max = 1). Use the Ctr struct for each counter, which manages all of this.
// See external Counter* methods for Python-safe single-return-value versions.
Counter(scale TimeScales) (cur, prv int, changed bool)
}
Env defines an interface for environments, which determine the nature and sequence of States that can be used as inputs to a model, and the Env also can accept Action responses from the model that affect state evolution.
The Env encapsulates all of the counter management logic to advance the temporal state of the environment, using TimeScales standard intervals.
State is comprised of one or more Elements, each of which consists of an etensor.Tensor chunk of values that can be obtained by the model. Likewise, Actions can also have Elements. The Step method is the main interface for advancing the Env state. Counters should be queried after calling Step to see if any relevant values have changed, to trigger functions in the model (e.g., logging of prior statistics, etc).
Typically each specific implementation of this Env interface will have multiple parameters etc that can be modified to control env behavior -- all of this is paradigm-specific and outside the scope of this basic interface.
type FixedTable ¶
type FixedTable struct {
Nm string `desc:"name of this environment"`
Dsc string `desc:"description of this environment"`
Table *etable.IdxView `` /* 285-byte string literal not displayed */
Sequential bool `` /* 140-byte string literal not displayed */
Order []int `desc:"permuted order of items to present if not sequential -- updated every time through the list"`
Run Ctr `view:"inline" desc:"current run of model as provided during Init"`
Epoch Ctr `view:"inline" desc:"number of times through entire set of patterns"`
Trial Ctr `` /* 164-byte string literal not displayed */
TrialName string `desc:"if Table has a Name column, this is the contents of that for current trial"`
PrvTrialName string `desc:"previous trial name"`
}
FixedTable is a basic Env that manages patterns from an etable.Table, with either sequential or permuted random ordering, and uses standard Trial / Epoch TimeScale counters to record progress and iterations through the table. It also records the outer loop of Run as provided by the model. It uses an IdxView indexed view of the Table, so a single shared table can be used across different environments, with each having its own unique view.
func (*FixedTable) Actions ¶
func (ft *FixedTable) Actions() Elements
func (*FixedTable) Counter ¶
func (ft *FixedTable) Counter(scale TimeScales) (cur, prv int, chg bool)
func (*FixedTable) Counters ¶
func (ft *FixedTable) Counters() []TimeScales
func (*FixedTable) Desc ¶
func (ft *FixedTable) Desc() string
func (*FixedTable) Init ¶
func (ft *FixedTable) Init(run int)
func (*FixedTable) Name ¶
func (ft *FixedTable) Name() string
func (*FixedTable) Row ¶
func (ft *FixedTable) Row() int
Row returns the current row number in table, based on Sequential / perumuted Order and already de-referenced through the IdxView's indexes to get the actual row in the table.
func (*FixedTable) SetTrialName ¶
func (ft *FixedTable) SetTrialName()
func (*FixedTable) States ¶
func (ft *FixedTable) States() Elements
func (*FixedTable) Step ¶
func (ft *FixedTable) Step() bool
func (*FixedTable) Validate ¶
func (ft *FixedTable) Validate() error
type TimeScales ¶
type TimeScales int32
TimeScales are the different time scales associated with overall simulation running, and can be used to parameterize the updating and control flow of simulations at different scales. The definitions become increasingly subjective imprecise as the time scales increase. Environments can implement updating along different such time scales as appropriate. This list is designed to standardize terminology across simulations and establish a common conceptual framework for time -- it can easily be extended in specific simulations to add needed additional levels, although using one of the existing standard values is recommended wherever possible.
const ( // Event is the smallest unit of naturalistic experience that coheres unto itself // (e.g., something that could be described in a sentence). // Typically this is on the time scale of a few seconds: e.g., reaching for // something, catching a ball. In an experiment it could just be the onset // of a stimulus, or the generation of a response. Event TimeScales = iota // Trial is one unit of behavior in an experiment, and could potentially // encompass multiple Events (e.g., one event is fixation, next is stimulus, // last is response, all comprising one Trial). It is also conventionally // used as a single Input / Output learning instance in a standard error-driven // learning paradigm. Trial // Sequence is a sequential group of Trials (not always needed). Sequence // Block is a collection of Trials, Sequences or Events, often used in experiments // when conditions are varied across blocks. Block // Epoch is used in two different contexts. In machine learning, it represents a // collection of Trials, Sequences or Events that constitute a "representative sample" // of the environment. In the simplest case, it is the entire collection of Trials // used for training. In electrophysiology, it is a timing window used for organizing // the analysis of electrode data. Epoch // Run is a complete run of a model / subject, from training to testing, etc. // Often multiple runs are done in an Expt to obtain statistics over initial // random weights etc. Run // Expt is an entire experiment -- multiple Runs through a given protocol / set of // parameters. Expt // Scene is a sequence of events that constitutes the next larger-scale coherent unit // of naturalistic experience corresponding e.g., to a scene in a movie. // Typically consists of events that all take place in one location over // e.g., a minute or so. This could be a paragraph or a page or so in a book. Scene // Episode is a sequence of scenes that constitutes the next larger-scale unit // of naturalistic experience e.g., going to the grocery store or eating at a // restaurant, attending a wedding or other "event". // This could be a chapter in a book. Episode TimeScalesN )
The time scales
func (*TimeScales) FromString ¶
func (i *TimeScales) FromString(s string) error
func (TimeScales) MarshalJSON ¶
func (ev TimeScales) MarshalJSON() ([]byte, error)
func (TimeScales) String ¶
func (i TimeScales) String() string
func (*TimeScales) UnmarshalJSON ¶
func (ev *TimeScales) UnmarshalJSON(b []byte) error
Source Files