armaze

package
v2.0.0-dev0.0.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 22, 2024 License: BSD-3-Clause Imports: 29 Imported by: 0

README

Arm Maze (armaze) Environment

The Arm Maze environment represents an N-armed maze ("bandit") with each Arm having a distinctive CS stimulus at the start (could be one of multiple possibilities) and (some probability of) a US outcome at the end of the maze (could be either positive or negative, with (variable) magnitude and probability.

It has a full 3D GUI showing the current state.

The controlling Sim is responsible for calling the NewStart() method when the agent should be placed back at the starting point. Action() records the action and updates the State rendering of the action, but doesn't update the environment until Step, so the environment at the end of a trial always reflects the current trial state.

Documentation

Overview

Package armaze represents an N-armed maze ("bandit") with each Arm having a distinctive CS stimulus at the start (could be one of multiple possibilities) and (some probability of) a US outcome at the end of the maze (could be either positive or negative, with (variable) magnitude and probability.

The maze can have open or closed arms -- open arms allow switching to a neighboring arm anytime, while closed arms only allow switching at the start.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func MinMaxRand 

func MinMaxRand(mm minmax.F32, rand randx.SysRand) float32

MinMaxRand returns a random number in the range between Min and Max

Types

type Actions 

type Actions int32 //enums:enum

Actions is a list of mutually exclusive states for tracing the behavior and internal state of Emery 

const (
	Forward Actions = iota
	Left
	Right
	Consume
	None
)
const ActionsN Actions = 5

ActionsN is the highest valid value for type Actions, plus one.

func ActionsValues 

func ActionsValues() []Actions

ActionsValues returns all possible values for the type Actions.

func (Actions) Desc 

func (i Actions) Desc() string

Desc returns the description of the Actions value.

func (Actions) Int64 

func (i Actions) Int64() int64

Int64 returns the Actions value as an int64.

func (Actions) MarshalText 

func (i Actions) MarshalText() ([]byte, error)

MarshalText implements the encoding.TextMarshaler interface.

func (*Actions) SetInt64 

func (i *Actions) SetInt64(in int64)

SetInt64 sets the Actions value from an int64.

func (*Actions) SetString 

func (i *Actions) SetString(s string) error

SetString sets the Actions value from its string representation, and returns an error if the string is invalid.

func (Actions) String 

func (i Actions) String() string

String returns the string representation of this Actions value.

func (*Actions) UnmarshalText 

func (i *Actions) UnmarshalText(text []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface.

func (Actions) Values 

func (i Actions) Values() []enums.Enum

Values returns all possible values for the type Actions.

type Arm 

type Arm struct {
	// CS == index of Arm
	CS int

	// length of arm: distance from CS start to US end for this arm
	Length int

	// index of US present at the end of this arm.
	// Indexes [0:NDrives] are positive USs, and beyond that are negative USs.
	US int

	// range of different effort levels per step (uniformly randomly sampled per step) for going down this arm
	Effort minmax.F32

	// range of different US magnitudes (uniformly sampled)
	USMag minmax.F32

	// probability of delivering the US
	USProb float32

	// USAvail indicates that the US is available on this trial: this is computed
	// from the USProb at the start of each behavioral trial
	USAvail bool `edit:"-"`

	// nominal expected value = US.Prob * US.Mag
	ExValue float32 `edit:"-"`

	// nominal expected cost = effort + normalized length
	ExCost float32 `edit:"-"`

	// nominal expected utility = ExValue - CostFactor * ExCost.
	// This is only meaningful relative to other options, not in any absolute terms.
	ExUtil float32 `edit:"-"`

	// UtilGroup is the group id for computing the BestOption utility for this arm:
	// = US for positive, and NDrives for all negative USs
	UtilGroup int

	// BestOption is true if this arm represents the best option in terms of ExUtil
	// relative to other options _for the same US_.
	// All negative USs are considered as one group for ranking.
	BestOption bool `edit:"-"`
}

Arm represents the properties of a given arm of the N-maze, representing a different choice option with different cost / benefit tradeoffs, in terms of distance and effort factors for getting down the arm, and US present at the end, which is delivered with a given probability and magnitude range. Each arm has its own distinctive CS visible at the start, which is the only cue used for the agent to decide whether to choose this arm or not.

func (*Arm) Defaults 

func (arm *Arm) Defaults()

func (*Arm) Empty 

func (arm *Arm) Empty()

Empty sets all state to -1

type Config 

type Config struct {

	// experimental paradigm that governs the configuration of environment based on params,
	// e.g., how the Range values are assigned to different arms.
	Paradigm Paradigms

	// for debugging, print out key steps including a trace of the action generation logic
	Debug bool

	// number of different drive-like body states (hunger, thirst, etc),
	// that are satisfied by a corresponding positive US outcome.
	// This is in addition to the first curiosity drive, which is always present.
	NDrives int

	// number of negative US outcomes -- these are added after NDrives positive USs to total US list
	NNegUSs int

	// total number of USs = NDrives + NNegUSs
	NUSs int `edit:"-"`

	// number of different arms, each of which has its own distinctive CS.
	// This is determined by the Paradigm (e.g., 2*NUSs for the Group cases).
	NArms int `edit:"-"`

	// range of arm length sallocated across arms, per Paradigm
	LengthRange minmax.Int `nest:"+"`

	// range of effort values allocated across arms, per Paradigm
	EffortRange minmax.F32 `nest:"+"`

	// range of US magnitudes allocated across arms, per Paradigm
	USMagRange minmax.F32 `nest:"+"`

	// range of US probabilities allocated across arms, per Paradigm
	USProbRange minmax.F32 `nest:"+"`

	// parameters for each arm option: dist, effort, US
	Arms []*Arm

	// misc params
	Params Params `view:"add-fields"`
}

Config has environment configuration

func (*Config) Defaults 

func (cfg *Config) Defaults()

func (*Config) Update 

func (cfg *Config) Update()

type Env 

type Env struct {

	// name of environment -- Train or Test
	Nm string

	// our data parallel index
	Di int `edit:"-"`

	// configuration parameters
	Config Config

	// current drive strength for each of Config.NDrives in normalized
	// 0-1 units of each drive (beyond built-in curiosity drive)
	Drives []float32

	// current arm location: either facing (Pos=0) or in (Pos > 0)
	Arm int `edit:"-"`

	// current position in the Arm: 0 = at start looking in, otherwise at given distance into the arm
	Pos int `edit:"-"`

	// distance from US within current arm
	Dist int `edit:"-"`

	// current integer time step since last NewStart
	Tick int `edit:"-"`

	// current target drive, in paradigms where that is used (e.g., Approach)
	TrgDrive int `edit:"-"`

	// Current US being consumed -- is -1 unless being consumed
	USConsumed int `edit:"-"`

	// reward or punishment value generated by the current US being consumed.
	// just the Magnitude of the US -- does NOT include any modulation by Drive
	USValue float32 `edit:"-"`

	// just finished consuming a US -- ready to start doing something new
	JustConsumed bool `edit:"-"`

	// last action taken
	LastAct Actions `edit:"-"`

	// effort on current trial
	Effort float32 `edit:"-"`

	// last CS seen
	LastCS int `edit:"-"`

	// last US -- previous trial
	LastUS int `edit:"-"`

	// true if looking at correct CS for first time
	ShouldGate bool `edit:"-"`

	// just gated on this trial -- set by sim-- used for instinct
	JustGated bool `edit:"-"`

	// has gated at some point during sequence -- set by sim -- used for instinct
	HasGated bool `edit:"-"`

	// named states -- e.g., USs, CSs, etc
	States map[string]*tensor.Float32

	// maximum length of any arm
	MaxLength int `edit:"-"`

	// random number generator for the env -- all random calls must use this
	Rand randx.SysRand `view:"-"`

	// random seed
	RandSeed int64 `edit:"-"`
}

Env implements an N-armed maze ("bandit") with each Arm having a distinctive CS stimulus visible at the start (could be one of multiple possibilities) and (some probability of) a US outcome at the end of the maze (could be either positive or negative, with (variable) magnitude and probability.

func (*Env) Action 

func (ev *Env) Action(action string, nop tensor.Tensor)

Action records the LastAct and renders it, but does not update the state accordingly.

func (*Env) ArmIsBest 

func (ev *Env) ArmIsBest(armIdx int) bool

ArmIsBest returns true if the given arm is the best choice, based on the arm being marked as BestChoice in terms of its general expected utility, and in terms of the US matching an active drive and not being a negative US outcome.

func (*Env) ArmIsNegative 

func (ev *Env) ArmIsNegative(armIdx int) bool

ArmIsNegative returns true if the given arm has a negative outcome

func (*Env) ConfigEnv 

func (ev *Env) ConfigEnv(di int)

ConfigEnv configures the environment. additional parameterization via specific configs is applied after this step, which initializes everything according to basic Ns takes the data parallel index di

func (*Env) ConfigGroupGoodBad 

func (ev *Env) ConfigGroupGoodBad()

ConfigGroupGoodBad

func (*Env) ConsumeEffort 

func (ev *Env) ConsumeEffort() float32

ConsumeEffort returns a new random Effort value from Config.Params.ConsumeEffort param range.

func (*Env) ConsumeUS 

func (ev *Env) ConsumeUS(arm *Arm)

ConsumeUS implements the consume action at current position in given arm

func (*Env) Counter 

func (ev *Env) Counter(scale env.TimeScales) (cur, prv int, changed bool)

func (*Env) CurArm 

func (ev *Env) CurArm() *Arm

CurArm returns current Arm

func (*Env) CurCS 

func (ev *Env) CurCS() int

CurCS returns current CS from current Arm

func (*Env) DecodeAct 

func (ev *Env) DecodeAct(vt *tensor.Float32) Actions

func (*Env) DecodeLocalist 

func (ev *Env) DecodeLocalist(vt *tensor.Float32) int

func (*Env) Defaults 

func (ev *Env) Defaults()

Defaults sets default params

func (*Env) Desc 

func (ev *Env) Desc() string

func (*Env) ExpectedUtilities 

func (ev *Env) ExpectedUtilities()

func (*Env) ForwardEffort 

func (ev *Env) ForwardEffort(arm *Arm) float32

ForwardEffort returns a new random Effort value from Arm Effort range

func (*Env) InactiveValue 

func (ev *Env) InactiveValue() float32

InactiveVal returns a new random inactive value from Config.Params.Inactive param range.

func (*Env) Init 

func (ev *Env) Init(run int)

Init does updating preparing to run -- params could have changed since initial config so updates everything except broad overall config stuff.

func (*Env) InstinctAct 

func (ev *Env) InstinctAct(justGated, hasGated bool) Actions

InstinctAct returns an "instinctive" action that implements a basic policy

func (*Env) Name 

func (ev *Env) Name() string

func (*Env) NewStart 

func (ev *Env) NewStart()

NewStart starts a new approach run

func (*Env) RenderAction 

func (ev *Env) RenderAction(act Actions)

RenderAction renders the action

func (*Env) RenderLocalist 

func (ev *Env) RenderLocalist(name string, val int)

RenderLocalist renders one localist state

func (*Env) RenderLocalist4D 

func (ev *Env) RenderLocalist4D(name string, val int)

RenderLocalist4D renders one localist state in 4D

func (*Env) RenderState 

func (ev *Env) RenderState()

RenderState renders the current state

func (*Env) State 

func (ev *Env) State(el string) tensor.Tensor

func (*Env) Step 

func (ev *Env) Step() bool

Step does one step. it is up to the driving sim to decide when to call NewStart

func (*Env) TakeAct 

func (ev *Env) TakeAct(act Actions)

func (*Env) TurnEffort 

func (ev *Env) TurnEffort() float32

TurnEffort returns a new random Effort value from Config.Params.TurnEffort param range.

func (*Env) UpdateMaxLength 

func (ev *Env) UpdateMaxLength()

func (*Env) Validate 

func (ev *Env) Validate() error

type GUI 

type GUI struct {

	// update display -- turn off to make it faster
	Disp bool

	// the env being visualized
	Env *Env

	// name of current env -- number is NData index
	EnvName string

	// 3D visualization of the Scene
	SceneView *xyzview.SceneView

	// 2D visualization of the Scene
	Scene2D *core.SVG

	// list of material colors
	MatColors []string

	// internal state colors
	StateColors map[string]string

	// thickness (X) and height (Y) of walls
	WallSize math32.Vector2

	// current internal / behavioral state
	State TraceStates

	// trace record of recent activity
	Trace StateTrace

	// view of the gui obj
	StructView *views.StructView `view:"-"`

	// ArmMaze TabView
	WorldTabs *core.Tabs `view:"-"`

	// ArmMaze is running
	IsRunning bool `view:"-"`

	// current depth map
	DepthValues []float32

	// offscreen render camera settings
	Camera world.Camera

	// color map to use for rendering depth map
	DepthMap views.ColorMapName

	// first-person right-eye full field view
	EyeRFullImage *core.Image `view:"-"`

	// first-person right-eye fovea view
	EyeRFovImage *core.Image `view:"-"`

	// depth map bitmap view
	DepthImage *core.Image `view:"-"`

	// plot of positive valence drives, active OFC US state, and reward
	USposPlot *plotview.PlotView

	// data for USPlot
	USposData *table.Table

	// plot of negative valence active OFC US state, and outcomes
	USnegPlot *plotview.PlotView

	// data for USPlot
	USnegData *table.Table

	// geometry of world
	Geom Geom

	// world
	World *physics.Group

	// 3D view of world
	View3D *world.View `view:"-"`

	// emer group
	Emery *physics.Group `view:"-"`

	// arms group
	Arms *physics.Group `view:"-"`

	// stims group
	Stims *physics.Group `view:"-"`

	// Right eye of emery
	EyeR physics.Body `view:"-"`

	// contacts from last step, for body
	Contacts physics.Contacts `view:"-"`
}

GUI renders multiple views of the flat world env

func (*GUI) AddFloor 

func (vw *GUI) AddFloor(par *physics.Group, name string) *physics.Group

AddFloor adds a floor

func (*GUI) ConfigArms 

func (vw *GUI) ConfigArms(par *physics.Group) *physics.Group

ConfigArms adds all the arms

func (*GUI) ConfigEmery 

func (vw *GUI) ConfigEmery(par *physics.Group, length float32) *physics.Group

ConfigEmery constructs a new Emery virtual hamster

func (*GUI) ConfigStims 

func (vw *GUI) ConfigStims(par *physics.Group, name string, width, height float32) *physics.Group

ConfigStims constructs stimuli: CSs, USs

func (*GUI) ConfigUSPlots 

func (vw *GUI) ConfigUSPlots()

func (*GUI) ConfigView3D 

func (vw *GUI) ConfigView3D(se *xyz.Scene)

ConfigView3D makes the 3D view

func (*GUI) ConfigWorld 

func (vw *GUI) ConfigWorld()

ConfigWorld constructs a new virtual physics world for flat world

func (*GUI) ConfigWorldGUI 

func (vw *GUI) ConfigWorldGUI(ev *Env) *core.Body

ConfigWorldGUI configures all the world view GUI elements pass an initial env to use for configuring

func (*GUI) ConfigWorldView 

func (vw *GUI) ConfigWorldView(tg *tensorview.TensorGrid)

func (*GUI) Consume 

func (vw *GUI) Consume()

func (*GUI) Forward 

func (vw *GUI) Forward()

func (*GUI) GrabEyeImg 

func (vw *GUI) GrabEyeImg()

GrabEyeImg takes a snapshot from the perspective of Emer's right eye

func (*GUI) Left 

func (vw *GUI) Left()

func (*GUI) Right 

func (vw *GUI) Right()

func (*GUI) SetEmeryPose 

func (vw *GUI) SetEmeryPose()

func (*GUI) UpdateStims 

func (vw *GUI) UpdateStims()

func (*GUI) UpdateWorld 

func (vw *GUI) UpdateWorld(ctx *axon.Context, ev *Env, net *axon.Network, state TraceStates)

func (*GUI) UpdateWorldGUI 

func (vw *GUI) UpdateWorldGUI()

func (*GUI) ViewDepth 

func (vw *GUI) ViewDepth(depth []float32)

ViewDepth updates depth bitmap with depth data

type Geom 

type Geom struct {

	// width of arm -- emery rodent is 1 unit wide
	ArmWidth float32 `default:"2"`

	// total space between arms, ends up being divided on either side
	ArmSpace float32 `default:"1"`

	// multiplier per unit arm length -- keep square with width
	LengthScale float32 `default:"2"`

	// thickness of walls, floor
	Thick float32 `default:"0.1"`

	// height of walls
	Height float32 `default:"0.2"`

	// width + space
	ArmWidthTot float32 `edit:"-"`

	// computed total depth, starts at 0 goes deep
	Depth float32 `edit:"-"`

	// computed total width
	Width float32 `edit:"-"`

	// half width for centering on 0 X
	HalfWidth float32 `edit:"-"`
}

Geom is overall geometry of the space

func (*Geom) Config 

func (ge *Geom) Config(nArms int, maxLen int)

func (*Geom) Pos 

func (ge *Geom) Pos(arm, pos int) (x, y float32)

pos returns the center position for given arm, position coordinate

type Paradigms 

type Paradigms int32 //enums:enum

Paradigms is a list of experimental paradigms that govern the configuration of the arms. 

const (
	// GroupGoodBad allocates Arms into 2 groups, with first group unambiguously Good
	// and the second Bad, using the Min, Max values of each Range parameter:
	// Length, Effort, USMag, USProb. Good has Min cost, Max US, and opposite for Bad.
	// This also aligns with the ordering of USs, such that negative USs are last.
	GroupGoodBad Paradigms = iota

	// GroupRisk allocates Arms into 2 groups with conflicting Cost and Benefit
	// tradeoffs, with the first group having Min cost and Min US, and the second
	// group having Max cost and Max US.
	GroupRisk
)
const ParadigmsN Paradigms = 2

ParadigmsN is the highest valid value for type Paradigms, plus one.

func ParadigmsValues 

func ParadigmsValues() []Paradigms

ParadigmsValues returns all possible values for the type Paradigms.

func (Paradigms) Desc 

func (i Paradigms) Desc() string

Desc returns the description of the Paradigms value.

func (Paradigms) Int64 

func (i Paradigms) Int64() int64

Int64 returns the Paradigms value as an int64.

func (Paradigms) MarshalText 

func (i Paradigms) MarshalText() ([]byte, error)

MarshalText implements the encoding.TextMarshaler interface.

func (*Paradigms) SetInt64 

func (i *Paradigms) SetInt64(in int64)

SetInt64 sets the Paradigms value from an int64.

func (*Paradigms) SetString 

func (i *Paradigms) SetString(s string) error

SetString sets the Paradigms value from its string representation, and returns an error if the string is invalid.

func (Paradigms) String 

func (i Paradigms) String() string

String returns the string representation of this Paradigms value.

func (*Paradigms) UnmarshalText 

func (i *Paradigms) UnmarshalText(text []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface.

func (Paradigms) Values 

func (i Paradigms) Values() []enums.Enum

Values returns all possible values for the type Paradigms.

type Params 

type Params struct {

	// effort for turning
	TurnEffort minmax.F32 `nest:"+" default:"{'Min':0.5, 'Max':0.5}"`

	// effort for consuming US
	ConsumeEffort minmax.F32 `nest:"+" default:"{'Min':0.5, 'Max':0.5}"`

	// an arbitrary scaling factor for costs relative to values,
	// used in computing the expected utility ExUtil for an arm.
	// These utilities are only useful for relative comparisons,
	// that go into computing the UtilRank, which should be used for evaluating
	// overall choices.
	CostFactor float32 `default:"0.2"`

	// threshold for considering a drive to be active; used in evaluating whether
	// an Arm choice is considered to be a good option.
	ActiveDriveThr float32 `default:"0.5"`

	// always turn left -- zoolander style -- reduces degrees of freedom in evaluating behavior
	AlwaysLeft bool `default:"true"`

	// after running down an Arm, a new random starting location is selected (otherwise same arm as last run)
	RandomStart bool `default:"true"`

	// if true, allow movement between arms just by going Left or Right.
	// Otherwise once past the start, no switching is allowed
	OpenArms bool `default:"true"`

	// strength of inactive inputs (e.g., Drives in Approach paradigm)
	Inactive minmax.F32 `nest:"+" default:"{'Min':0, 'Max':0}" view:"inline"`

	// number of Y-axis repetitions of localist stimuli -- for redundancy in spiking nets
	NYReps int `default:"4"`
}

Params are misc environment parameters

type StateTrace 

type StateTrace []*TraceRec

StateTrace holds trace records

func (*StateTrace) AddRec 

func (tr *StateTrace) AddRec(ctx *axon.Context, di uint32, ev *Env, net *axon.Network, state TraceStates) *TraceRec

AddRec adds a record with data from given sources

type TraceRec 

type TraceRec struct {

	// absolute time
	Time float32

	// trial counter
	Trial int

	// current arm
	Arm int

	// position in arm
	Pos int

	// behavioral / internal state summary
	State TraceStates

	// NDrives current drive state level
	Drives []float32
}

TraceRec holds record of info for tracing behavior, state

type TraceStates 

type TraceStates int32 //enums:enum

TraceStates is a list of mutually exclusive states for tracing the behavior and internal state of Emery 

const (
	// Searching is not yet goal engaged, looking for a goal
	TrSearching TraceStates = iota

	// Deciding is having some partial gating but not in time for action
	TrDeciding

	// JustEngaged means just decided to engage in a goal
	TrJustEngaged

	// Approaching is goal engaged, approaching the goal
	TrApproaching

	// Consuming is consuming the US, first step (prior to getting reward, step1)
	TrConsuming

	// Rewarded is just received reward from a US
	TrRewarded

	// GiveUp is when goal is abandoned
	TrGiveUp

	// Bumping is bumping into a wall
	TrBumping
)
const TraceStatesN TraceStates = 8

TraceStatesN is the highest valid value for type TraceStates, plus one.

func TraceStatesValues 

func TraceStatesValues() []TraceStates

TraceStatesValues returns all possible values for the type TraceStates.

func (TraceStates) Desc 

func (i TraceStates) Desc() string

Desc returns the description of the TraceStates value.

func (TraceStates) Int64 

func (i TraceStates) Int64() int64

Int64 returns the TraceStates value as an int64.

func (TraceStates) MarshalText 

func (i TraceStates) MarshalText() ([]byte, error)

MarshalText implements the encoding.TextMarshaler interface.

func (*TraceStates) SetInt64 

func (i *TraceStates) SetInt64(in int64)

SetInt64 sets the TraceStates value from an int64.

func (*TraceStates) SetString 

func (i *TraceStates) SetString(s string) error

SetString sets the TraceStates value from its string representation, and returns an error if the string is invalid.

func (TraceStates) String 

func (i TraceStates) String() string

String returns the string representation of this TraceStates value.

func (*TraceStates) UnmarshalText 

func (i *TraceStates) UnmarshalText(text []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface.

func (TraceStates) Values 

func (i TraceStates) Values() []enums.Enum

Values returns all possible values for the type TraceStates.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.