Documentation ¶
Overview ¶
Package armaze represents an N-armed maze ("bandit") with each Arm having a distinctive CS stimulus at the start (could be one of multiple possibilities) and (some probability of) a US outcome at the end of the maze (could be either positive or negative, with (variable) magnitude and probability.
The maze can have open or closed arms -- open arms allow switching to a neighboring arm anytime, while closed arms only allow switching at the start.
Index ¶
- Variables
- func MinMaxRand(mm minmax.F32, rand erand.SysRand) float32
- type Actions
- type Arm
- type Config
- type Env
- func (ev *Env) Action(action string, nop etensor.Tensor)
- func (ev *Env) ArmIsMaxUtil(arm int) bool
- func (ev *Env) ArmIsMaxValue(arm int) bool
- func (ev *Env) ArmIsNegative(arm int) bool
- func (ev *Env) ChooseCSs()
- func (ev *Env) ConfigApproach()
- func (ev *Env) ConfigEnv(di int)
- func (ev *Env) ConsumeEffort() float32
- func (ev *Env) ConsumeUS(arm *Arm)
- func (ev *Env) Counter(scale env.TimeScales) (cur, prv int, changed bool)
- func (ev *Env) CurArm() *Arm
- func (ev *Env) CurCS() int
- func (ev *Env) DecodeAct(vt *etensor.Float32) Actions
- func (ev *Env) DecodeLocalist(vt *etensor.Float32) int
- func (ev *Env) Defaults()
- func (ev *Env) Desc() string
- func (ev *Env) ExValueUtil(pv *axon.PVLV, ctx *axon.Context)
- func (ev *Env) ForwardEffort(arm *Arm) float32
- func (ev *Env) InactiveVal() float32
- func (ev *Env) Init(run int)
- func (ev *Env) InstinctAct(justGated, hasGated bool) Actions
- func (ev *Env) Name() string
- func (ev *Env) NewStart()
- func (ev *Env) RenderAction(act Actions)
- func (ev *Env) RenderLocalist(name string, val int)
- func (ev *Env) RenderLocalist4D(name string, val int)
- func (ev *Env) RenderState()
- func (ev *Env) StartApproach()
- func (ev *Env) State(el string) etensor.Tensor
- func (ev *Env) Step() bool
- func (ev *Env) StepApproach()
- func (ev *Env) TakeAct(act Actions)
- func (ev *Env) TurnEffort() float32
- func (ev *Env) UpdateMaxLength()
- func (ev *Env) Validate() error
- type GUI
- func (vw *GUI) AddFloor(par *eve.Group, name string) *eve.Group
- func (vw *GUI) ConfigArms(par *eve.Group) *eve.Group
- func (vw *GUI) ConfigEmery(par *eve.Group, length float32) *eve.Group
- func (vw *GUI) ConfigStims(par *eve.Group, name string, width, height float32) *eve.Group
- func (vw *GUI) ConfigUSPlots()
- func (vw *GUI) ConfigView3D(sc *gi3d.Scene)
- func (vw *GUI) ConfigWorld()
- func (vw *GUI) ConfigWorldGUI(ev *Env) *gi.Window
- func (vw *GUI) ConfigWorldView(tg *etview.TensorGrid)
- func (vw *GUI) Consume()
- func (vw *GUI) Forward()
- func (vw *GUI) GrabEyeImg()
- func (vw *GUI) Left()
- func (vw *GUI) Right()
- func (vw *GUI) SetEmeryPose()
- func (vw *GUI) UpdateStims()
- func (vw *GUI) UpdateWorld(ctx *axon.Context, ev *Env, net *axon.Network, state TraceStates)
- func (vw *GUI) UpdateWorldGUI()
- func (vw *GUI) ViewDepth(depth []float32)
- type Geom
- type Paradigms
- type Params
- type StateTrace
- type TraceRec
- type TraceStates
- type USParams
Constants ¶
This section is empty.
Variables ¶
var KiT_Actions = kit.Enums.AddEnum(ActionsN, kit.NotBitFlag, nil)
var KiT_Paradigms = kit.Enums.AddEnum(ParadigmsN, kit.NotBitFlag, nil)
var KiT_TraceStates = kit.Enums.AddEnum(TraceStatesN, kit.NotBitFlag, nil)
Functions ¶
Types ¶
type Actions ¶
type Actions int
Actions is a list of mutually exclusive states for tracing the behavior and internal state of Emery
func (*Actions) FromString ¶
type Arm ¶
type Arm struct { // length of arm: distance from CS start to US end for this arm Length int `desc:"length of arm: distance from CS start to US end for this arm"` // range of different effort levels per step (uniformly randomly sampled per step) for going down this arm Effort minmax.F32 `desc:"range of different effort levels per step (uniformly randomly sampled per step) for going down this arm"` // index of US present at the end of this arm -- -1 if none US int `desc:"index of US present at the end of this arm -- -1 if none"` // index of CS visible at the start of this arm, -1 if none CS int `desc:"index of CS visible at the start of this arm, -1 if none"` // current expected value = US.Prob * US.Mag * Drives[US] -- computed at start of new approach ExValue float32 `inactive:"+" desc:"current expected value = US.Prob * US.Mag * Drives[US] -- computed at start of new approach"` // current expected PVpos value = normalized ExValue -- computed at start of new approach ExPVpos float32 `inactive:"+" desc:"current expected PVpos value = normalized ExValue -- computed at start of new approach"` // current expected PVneg value = normalized time and effort costs ExPVneg float32 `inactive:"+" desc:"current expected PVneg value = normalized time and effort costs"` // current expected utility = effort discounted version of ExPVpos -- computed at start of new approach ExUtil float32 `inactive:"+" desc:"current expected utility = effort discounted version of ExPVpos -- computed at start of new approach"` }
Arm represents the properties of a given arm of the N-maze. Arms have characteristic distance and effort factors for getting down the arm, and typically have a distinctive CS visible at the start and a US at the end, which has US-specific parameters for actually delivering reward or punishment.
type Config ¶
type Config struct { // experimental paradigm that governs the configuration and updating of environment state over time and the appropriate evaluation criteria. Paradigm Paradigms `` /* 144-byte string literal not displayed */ // for debugging, print out key steps including a trace of the action generation logic Debug bool `desc:"for debugging, print out key steps including a trace of the action generation logic"` // number of different drive-like body states (hunger, thirst, etc), that are satisfied by a corresponding positive US outcome -- this does not include the first curiosity drive NDrives int `` /* 181-byte string literal not displayed */ // number of negative US outcomes -- these are added after NDrives positive USs to total US list NNegUSs int `desc:"number of negative US outcomes -- these are added after NDrives positive USs to total US list"` // total number of USs = NDrives + NNegUSs NUSs int `inactive:"+" desc:"total number of USs = NDrives + NNegUSs"` // number of different arms NArms int `desc:"number of different arms"` // number of different CSs -- typically at least a unique CS per US -- relationship is determined in the US params NCSs int `desc:"number of different CSs -- typically at least a unique CS per US -- relationship is determined in the US params"` // [view: add-fields] misc params Params Params `view:"add-fields" desc:"misc params"` }
Config has environment configuration
type Env ¶
type Env struct { // name of environment -- Train or Test Nm string `desc:"name of environment -- Train or Test"` // our data parallel index Di int `inactive:"+" desc:"our data parallel index"` // configuration parameters Config Config `desc:"configuration parameters"` // current drive strength for each of Config.NDrives in normalized 0-1 units of each drive: 0 = first sim drive, not curiosity Drives []float32 `` /* 130-byte string literal not displayed */ // parameters associated with each US. The first NDrives are positive USs, and beyond that are negative USs USs []*USParams `desc:"parameters associated with each US. The first NDrives are positive USs, and beyond that are negative USs"` // state of each arm: dist, effort, US, CS Arms []*Arm `desc:"state of each arm: dist, effort, US, CS"` // arm-wise location: either facing (Pos=0) or in (Pos > 0) Arm int `inactive:"+" desc:"arm-wise location: either facing (Pos=0) or in (Pos > 0)"` // current position in the Arm: 0 = at start looking in, otherwise at given distance into the arm Pos int `inactive:"+" desc:"current position in the Arm: 0 = at start looking in, otherwise at given distance into the arm"` // current integer time step since last NewStart Tick int `inactive:"+" desc:"current integer time step since last NewStart"` // current target drive, in paradigms where that is used (e.g., Approach) TrgDrive int `inactive:"+" desc:"current target drive, in paradigms where that is used (e.g., Approach)"` // Current US being consumed -- is -1 unless being consumed USConsumed int `inactive:"+" desc:"Current US being consumed -- is -1 unless being consumed"` // reward or punishment value generated by the current US being consumed -- just the Magnitude of the US -- does NOT include any modulation by Drive USValue float32 `` /* 165-byte string literal not displayed */ // just finished consuming a US -- ready to start doing something new JustConsumed bool `inactive:"+" desc:"just finished consuming a US -- ready to start doing something new"` // arm(s) with maximum Drive * Mag * Prob US outcomes ArmsMaxValue []int `inactive:"+" desc:"arm(s) with maximum Drive * Mag * Prob US outcomes"` // maximum value for ArmsMaxValue arms MaxValue float32 `inactive:"+" desc:"maximum value for ArmsMaxValue arms"` // arm(s) with maximum Value outcome discounted by Effort ArmsMaxUtil []int `inactive:"+" desc:"arm(s) with maximum Value outcome discounted by Effort"` // maximum utility for ArmsMaxUtil arms MaxUtil float32 `inactive:"+" desc:"maximum utility for ArmsMaxUtil arms"` // arm(s) with negative US outcomes ArmsNeg []int `inactive:"+" desc:"arm(s) with negative US outcomes"` // last action taken LastAct Actions `inactive:"+" desc:"last action taken"` // effort on current trial Effort float32 `inactive:"+" desc:"effort on current trial"` // last CS seen LastCS int `inactive:"+" desc:"last CS seen"` // last US -- previous trial LastUS int `inactive:"+" desc:"last US -- previous trial"` // true if looking at correct CS for first time ShouldGate bool `inactive:"+" desc:"true if looking at correct CS for first time"` // just gated on this trial -- set by sim-- used for instinct JustGated bool `inactive:"+" desc:"just gated on this trial -- set by sim-- used for instinct"` // has gated at some point during sequence -- set by sim -- used for instinct HasGated bool `inactive:"+" desc:"has gated at some point during sequence -- set by sim -- used for instinct"` // named states -- e.g., USs, CSs, etc States map[string]*etensor.Float32 `desc:"named states -- e.g., USs, CSs, etc"` // maximum length of any arm MaxLength int `inactive:"+" desc:"maximum length of any arm"` // [view: -] random number generator for the env -- all random calls must use this Rand erand.SysRand `view:"-" desc:"random number generator for the env -- all random calls must use this"` // random seed RndSeed int64 `inactive:"+" desc:"random seed"` }
Env implements an N-armed maze ("bandit") with each Arm having a distinctive CS stimulus visible at the start (could be one of multiple possibilities) and (some probability of) a US outcome at the end of the maze (could be either positive or negative, with (variable) magnitude and probability.
func (*Env) Action ¶
Action records the LastAct and renders it, but does not update the state accordingly.
func (*Env) ArmIsMaxUtil ¶
ArmIsMaxUtil returns true if the given arm is (one of) the arms with the best current expected outcome utility
func (*Env) ArmIsMaxValue ¶
ArmIsMaxValue returns true if the given arm is (one of) the arms with the best current expected outcome value
func (*Env) ArmIsNegative ¶
ArmIsNegative returns true if the given arm is (one of) the arms with negative outcomes
func (*Env) ChooseCSs ¶
func (ev *Env) ChooseCSs()
ChooseCSs selects new CSs for each Arm as function of US CSProbs This must be called
func (*Env) ConfigApproach ¶
func (ev *Env) ConfigApproach()
ConfigApproach does initial config for Approach paradigm
func (*Env) ConfigEnv ¶
ConfigEnv configures the environment. additional parameterization via specific configs is applied after this step, which initializes everything according to basic Ns takes the data parallel index di
func (*Env) ConsumeEffort ¶
ConsumeEffort returns a new random Effort value from Config.Params.ConsumeEffort param range.
func (*Env) ForwardEffort ¶
ForwardEffort returns a new random Effort value from Arm Effort range
func (*Env) InactiveVal ¶
InactiveVal returns a new random inactive value from Config.Params.Inactive param range.
func (*Env) Init ¶
Init does updating preparing to run -- params could have changed since initial config so updates everything except broad overall config stuff.
func (*Env) InstinctAct ¶
InstinctAct returns an "instinctive" action that implements a basic policy
func (*Env) RenderLocalist ¶
RenderLocalist renders one localist state
func (*Env) RenderLocalist4D ¶
RenderLocalist4D renders one localist state in 4D
func (*Env) StartApproach ¶
func (ev *Env) StartApproach()
StartApproach does new start state setting for Approach Selects a new TrgDrive at random, sets that to 1, others to inactive levels
func (*Env) StepApproach ¶
func (ev *Env) StepApproach()
func (*Env) TurnEffort ¶
TurnEffort returns a new random Effort value from Config.Params.TurnEffort param range.
func (*Env) UpdateMaxLength ¶
func (ev *Env) UpdateMaxLength()
type GUI ¶
type GUI struct { // update display -- turn off to make it faster Disp bool `desc:"update display -- turn off to make it faster"` // the env being visualized Env *Env `desc:"the env being visualized"` // name of current env -- number is NData index EnvName string `desc:"name of current env -- number is NData index"` // list of material colors MatColors []string `desc:"list of material colors"` // internal state colors StateColors map[string]string `desc:"internal state colors"` // thickness (X) and height (Y) of walls WallSize mat32.Vec2 `desc:"thickness (X) and height (Y) of walls"` // current internal / behavioral state State TraceStates `desc:"current internal / behavioral state"` // trace record of recent activity Trace StateTrace `desc:"trace record of recent activity"` // [view: -] view of the gui obj StructView *giv.StructView `view:"-" desc:"view of the gui obj"` // [view: -] ArmMaze GUI window WorldWin *gi.Window `view:"-" desc:"ArmMaze GUI window"` // [view: -] ArmMaze TabView WorldTabs *gi.TabView `view:"-" desc:"ArmMaze TabView"` // [view: -] ArmMaze is running IsRunning bool `view:"-" desc:"ArmMaze is running"` // current depth map DepthVals []float32 `desc:"current depth map"` // offscreen render camera settings Camera evev.Camera `desc:"offscreen render camera settings"` // color map to use for rendering depth map DepthMap giv.ColorMapName `desc:"color map to use for rendering depth map"` // [view: -] first-person right-eye full field view EyeRFullImg *gi.Bitmap `view:"-" desc:"first-person right-eye full field view"` // [view: -] first-person right-eye fovea view EyeRFovImg *gi.Bitmap `view:"-" desc:"first-person right-eye fovea view"` // [view: -] depth map bitmap view DepthImg *gi.Bitmap `view:"-" desc:"depth map bitmap view"` // plot of positive valence drives, active OFC US state, and reward USposPlot *eplot.Plot2D `desc:"plot of positive valence drives, active OFC US state, and reward"` // data for USPlot USposData *etable.Table `desc:"data for USPlot"` // plot of negative valence active OFC US state, and outcomes USnegPlot *eplot.Plot2D `desc:"plot of negative valence active OFC US state, and outcomes"` // data for USPlot USnegData *etable.Table `desc:"data for USPlot"` // geometry of world Geom Geom `desc:"geometry of world"` // world World *eve.Group `desc:"world"` // [view: -] 3D view of world View3D *evev.View `view:"-" desc:"3D view of world"` // [view: -] emer group Emery *eve.Group `view:"-" desc:"emer group"` // [view: -] arms group Arms *eve.Group `view:"-" desc:"arms group"` // [view: -] stims group Stims *eve.Group `view:"-" desc:"stims group"` // [view: -] Right eye of emery EyeR eve.Body `view:"-" desc:"Right eye of emery"` // [view: -] contacts from last step, for body Contacts eve.Contacts `view:"-" desc:"contacts from last step, for body"` // [view: -] gui window Win *gi.Window `view:"-" desc:"gui window"` }
GUI renders multiple views of the flat world env
func (*GUI) ConfigArms ¶
ConfigArms adds all the arms
func (*GUI) ConfigEmery ¶
ConfigEmery constructs a new Emery virtual hamster
func (*GUI) ConfigStims ¶
ConfigStims constructs stimuli: CSs, USs
func (*GUI) ConfigUSPlots ¶
func (vw *GUI) ConfigUSPlots()
func (*GUI) ConfigView3D ¶
ConfigView3D makes the 3D view
func (*GUI) ConfigWorld ¶
func (vw *GUI) ConfigWorld()
ConfigWorld constructs a new virtual physics world for flat world
func (*GUI) ConfigWorldGUI ¶
ConfigWorldGUI configures all the world view GUI elements pass an initial env to use for configuring
func (*GUI) ConfigWorldView ¶
func (vw *GUI) ConfigWorldView(tg *etview.TensorGrid)
func (*GUI) GrabEyeImg ¶
func (vw *GUI) GrabEyeImg()
GrabEyeImg takes a snapshot from the perspective of Emer's right eye
func (*GUI) SetEmeryPose ¶
func (vw *GUI) SetEmeryPose()
func (*GUI) UpdateStims ¶
func (vw *GUI) UpdateStims()
func (*GUI) UpdateWorld ¶
func (*GUI) UpdateWorldGUI ¶
func (vw *GUI) UpdateWorldGUI()
type Geom ¶
type Geom struct { // [def: 2] width of arm -- emery rodent is 1 unit wide ArmWidth float32 `def:"2" desc:"width of arm -- emery rodent is 1 unit wide"` // [def: 1] total space between arms, ends up being divided on either side ArmSpace float32 `def:"1" desc:"total space between arms, ends up being divided on either side"` // [def: 2] multiplier per unit arm length -- keep square with width LengthScale float32 `def:"2" desc:"multiplier per unit arm length -- keep square with width"` // [def: 0.1] thickness of walls, floor Thick float32 `def:"0.1" desc:"thickness of walls, floor"` // [def: 0.2] height of walls Height float32 `def:"0.2" desc:"height of walls"` // width + space ArmWidthTot float32 `inactive:"+" desc:"width + space"` // computed total depth, starts at 0 goes deep Depth float32 `inactive:"+" desc:"computed total depth, starts at 0 goes deep"` // computed total width Width float32 `inactive:"+" desc:"computed total width"` // half width for centering on 0 X HalfWidth float32 `inactive:"+" desc:"half width for centering on 0 X"` }
Geom is overall geometry of the space
type Paradigms ¶
type Paradigms int
Paradigms is a list of experimental paradigms that govern the configuration and updating of environment state over time and the appropriate evaluation criteria.
const ( // Approach is a basic case where one Drive (chosen at random each trial) is fully active and others are at InactiveDrives levels -- goal is to approach the CS associated with the Drive-satisfying US, and avoid negative any negative USs. USs are always placed in same Arms (NArms must be >= NUSs -- any additional Arms are filled at random with additional US copies) Approach Paradigms = iota ParadigmsN )
type Params ¶
type Params struct { // [def: {'Min':0.5, 'Max':0.5}] effort for turning TurnEffort minmax.F32 `def:"{'Min':0.5, 'Max':0.5}" desc:"effort for turning"` // [def: {'Min':0.5, 'Max':0.5}] effort for consuming US ConsumeEffort minmax.F32 `def:"{'Min':0.5, 'Max':0.5}" desc:"effort for consuming US"` // [def: true] always turn left -- zoolander style -- reduces degrees of freedom in evaluating behavior AlwaysLeft bool `def:"true" desc:"always turn left -- zoolander style -- reduces degrees of freedom in evaluating behavior"` // [def: false] permute the order of USs prior to applying them to arms -- having this off makes it easier to visually determine match between Drive and arm approach, and shouldn't make any difference to behavior (model doesn't know about this ordering). PermuteUSs bool `` /* 257-byte string literal not displayed */ // [def: true] evenly distribute CSs among the USs. Otherwise the configuration must explicitly assign probabilities within each US for each CS. EvenCSs bool `` /* 148-byte string literal not displayed */ // [def: true] after running down an Arm, a new random starting location is selected (otherwise same arm as last run) RandomStart bool `def:"true" desc:"after running down an Arm, a new random starting location is selected (otherwise same arm as last run)"` // [def: true] if true, allow movement between arms just by going Left or Right -- otherwise once past the start, no switching is allowed OpenArms bool `` /* 140-byte string literal not displayed */ // [def: {'Min':0, 'Max':0}] [view: inline] strength of inactive inputs (e.g., Drives in Approach paradigm) Inactive minmax.F32 `def:"{'Min':0, 'Max':0}" view:"inline" desc:"strength of inactive inputs (e.g., Drives in Approach paradigm)"` // [def: 4] number of Y-axis repetitions of localist stimuli -- for redundancy in spiking nets NYReps int `def:"4" desc:"number of Y-axis repetitions of localist stimuli -- for redundancy in spiking nets"` }
Params are misc environment parameters
type TraceRec ¶
type TraceRec struct { // absolute time Time float32 `desc:"absolute time"` // trial counter Trial int `desc:"trial counter"` // current arm Arm int `desc:"current arm"` // position in arm Pos int `desc:"position in arm"` // behavioral / internal state summary State TraceStates `desc:"behavioral / internal state summary"` // NDrives current drive state level Drives []float32 `desc:"NDrives current drive state level"` }
TraceRec holds record of info for tracing behavior, state
type TraceStates ¶
type TraceStates int
TraceStates is a list of mutually exclusive states for tracing the behavior and internal state of Emery
const ( // Searching is not yet goal engaged, looking for a goal TrSearching TraceStates = iota // Deciding is having some partial gating but not in time for action TrDeciding // JustEngaged means just decided to engage in a goal TrJustEngaged // Approaching is goal engaged, approaching the goal TrApproaching // Consuming is consuming the US, first step (prior to getting reward, step1) TrConsuming // Rewarded is just received reward from a US TrRewarded // GiveUp is when goal is abandoned TrGiveUp // Bumping is bumping into a wall TrBumping TraceStatesN )
func (TraceStates) Desc ¶
func (i TraceStates) Desc() string
func (*TraceStates) FromString ¶
func (i *TraceStates) FromString(s string) error
func (TraceStates) String ¶
func (i TraceStates) String() string
type USParams ¶
type USParams struct { // if true is a negative valence US -- these are after the first NDrives USs Negative bool `desc:"if true is a negative valence US -- these are after the first NDrives USs"` // range of different magnitudes (uniformly sampled) Mag minmax.F32 `desc:"range of different magnitudes (uniformly sampled)"` // probability of delivering the US Prob float32 `desc:"probability of delivering the US"` // probabilities of each CS being active for this US CSProbs []float32 `desc:"probabilities of each CS being active for this US"` }
USParams has parameters for different USs