README
¶
FFFB Inhibition
FFFB is the feedforward (FF) and feedback (FB) inhibition mechanism, originally developed for the Leabra model. It produces a robust, graded k-Winners-Take-All dynamic of sparse distributed representations having approximately k out of N neurons active at any time, where k is typically 10-20 percent of N.
-
FF is a simple linear function of the average netinput (Ge) coming into neurons in a layer / pool. It is critical to have a
FF0offset for the zero-point for FF inhibition. Note that by giving FF access to the netinput, it implicitly has access to the strength of synaptic weights in a layer, which makes it automatically more robust over the course of learning. -
FB must be integrated over time to avoid oscillations, but is otherwise also a weighted proportion of average activity in a layer.
Documentation
¶
Overview ¶
Package fffb provides feedforward (FF) and feedback (FB) inhibition (FFFB) based on average (or maximum) excitatory Ge (FF) and activation (FB).
This produces a robust, graded k-Winners-Take-All dynamic of sparse distributed representations having approximately k out of N neurons active at any time, where k is typically 10-20 percent of N.
Index ¶
- type Bg
- type Inhib
- type Inhibs
- type Params
- func (fb *Params) Defaults()
- func (fb *Params) FBInhib(avgAct float32) float32
- func (fb *Params) FBUpdate(fbi *float32, newFbi float32)
- func (fb *Params) FFInhib(avgGe, maxGe float32) float32
- func (fb *Params) Inhib(inh *Inhib, gimult float32)
- func (fb *Params) ShouldDisplay(field string) bool
- func (fb *Params) Update()
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Bg ¶
type Bg struct {
// enable adaptive layer inhibition gain as stored in layer GiCur value
On bool
// level of inhibition as proporition of FFFB Gi value -- will need to reduce FFFB level to compensate for this additional source of inhibition
Gi float32 `default:".1"`
// time constant for integrating background inhibition (tau is roughly how long it takes for value to change significantly -- 1.4x the half-life)
Tau float32 `default:"10"`
// rate = 1 / tau
Dt float32 `edit:"-" display:"-" json:"-" xml:"-"`
}
Bg has parameters for a slower, low level of background inhibition based on main FFFB computed inhibition.
func (*Bg) ShouldDisplay ¶
type Inhib ¶
type Inhib struct {
// computed feedforward inhibition
FFi float32
// computed feedback inhibition (total)
FBi float32
// overall value of the FFFB computed inhibition -- this is what is added into the unit Gi inhibition level (along with GiBg and any synaptic unit-driven inhibition)
Gi float32
// original value of the inhibition (before pool or other effects)
GiOrig float32
// for pools, this is the layer-level inhibition that is MAX'd with the pool-level inhibition to produce the net inhibition
LayGi float32
// average and max Ge excitatory conductance values, which drive FF inhibition
Ge minmax.AvgMax32
// average and max Act activation values, which drive FB inhibition
Act minmax.AvgMax32
}
Inhib contains state values for computed FFFB inhibition
type Params ¶
type Params struct {
// enable this level of inhibition
On bool
// overall inhibition gain -- this is main parameter to adjust to change overall activation levels -- it scales both the the ff and fb factors uniformly
Gi float32 `min:"0" default:"1.1"`
// overall inhibitory contribution from feedforward inhibition -- multiplies average Ge (i.e., synaptic drive into layer) -- this anticipates upcoming changes in excitation, but if set too high, it can make activity slow to emerge -- see also ff0 for a zero-point for this value
FF float32 `min:"0" default:"1"`
// overall inhibitory contribution from feedback inhibition -- multiplies average activation -- this reacts to layer activation levels and works more like a thermostat (turning up when the 'heat' in the layer is too high)
FB float32 `min:"0" default:"1"`
// time constant in cycles, which should be milliseconds typically (tau is roughly how long it takes for value to change significantly -- 1.4x the half-life) for integrating feedback inhibitory values -- prevents oscillations that otherwise occur -- the fast default of 1.4 should be used for most cases but sometimes a slower value (3 or higher) can be more robust, especially when inhibition is strong or inputs are more rapidly changing
FBTau float32 `min:"0" default:"1.4,3,5"`
// what proportion of the maximum vs. average Ge to use in the feedforward inhibition computation -- 0 = all average, 1 = all max, and values in between = proportional mix between average and max (ff_netin = avg + ff_max_vs_avg * (max - avg)) -- including more max can be beneficial especially in situations where the average can vary significantly but the activity should not -- max is more robust in many situations but less flexible and sensitive to the overall distribution -- max is better for cases more closely approximating single or strictly fixed winner-take-all behavior -- 0.5 is a good compromise in many cases and generally requires a reduction of .1 or slightly more (up to .3-.5) from the gi value for 0
MaxVsAvg float32 `default:"0,0.5,1"`
// feedforward zero point for average Ge -- below this level, no FF inhibition is computed based on avg Ge, and this value is subtraced from the ff inhib contribution above this value -- the 0.1 default should be good for most cases (and helps FF_FB produce k-winner-take-all dynamics), but if average Ges are lower than typical, you may need to lower it
FF0 float32 `default:"0.1"`
// extra feedforward inhibition applied when average Ge exceeds a higher threshold -- produces a nonlinear inhibition effect that is consistent with a wide range of neuroscience data, including popout and the Reynolds & Heeger, 2009 attention model
FFEx float32 `default:"0,0.05"`
// point of average Ge at which extra inhibition based on feedforward level starts
FFEx0 float32 `default:"0.15"`
// rate = 1 / tau
FBDt float32 `edit:"-" display:"-" json:"-" xml:"-"`
}
Params parameterizes feedforward (FF) and feedback (FB) inhibition (FFFB) based on average (or maximum) Ge (FF) and activation (FB)
func (*Params) FBInhib ¶
FBInhib computes feedback inhibition value as function of average activation
func (*Params) FFInhib ¶
FFInhib returns the feedforward inhibition value based on average and max excitatory conductance within relevant scope
func (*Params) Inhib ¶
Inhib is full inhibition computation for given inhib state, which must have the Ge and Act values updated to reflect the current Avg and Max of those values in relevant inhibitory pool.
Source Files