The highest tagged major version is
README
¶
FFFB Inhibition
FFFB is the feedforward (FF) and feedback (FB) inhibition mechanism, originally developed for the Leabra model. It produces a robust, graded k-Winners-Take-All dynamic of sparse distributed representations having approximately k out of N neurons active at any time, where k is typically 10-20 percent of N.
-
FF is a simple linear function of the average netinput (Ge) coming into neurons in a layer / pool. It is critical to have a
FF0offset for the zero-point for FF inhibition. Note that by giving FF access to the netinput, it implicitly has access to the strength of synaptic weights in a layer, which makes it automatically more robust over the course of learning. -
FB must be integrated over time to avoid oscillations, but is otherwise also a weighted proportion of average activity in a layer.
Documentation
¶
Overview ¶
Package fffb provides feedforward (FF) and feedback (FB) inhibition (FFFB) based on average (or maximum) excitatory Ge (FF) and activation (FB).
This produces a robust, graded k-Winners-Take-All dynamic of sparse distributed representations having approximately k out of N neurons active at any time, where k is typically 10-20 percent of N.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Bg ¶ added in v1.2.33
type Bg struct {
// enable adaptive layer inhibition gain as stored in layer GiCur value
On bool `desc:"enable adaptive layer inhibition gain as stored in layer GiCur value"`
// [def: .1] [viewif: On=true] level of inhibition as proporition of FFFB Gi value -- will need to reduce FFFB level to compensate for this additional source of inhibition
Gi float32 `` /* 173-byte string literal not displayed */
// [def: 10] [viewif: On=true] time constant for integrating background inhibition (tau is roughly how long it takes for value to change significantly -- 1.4x the half-life)
Tau float32 `` /* 175-byte string literal not displayed */
// [view: -] rate = 1 / tau
Dt float32 `inactive:"+" view:"-" json:"-" xml:"-" desc:"rate = 1 / tau"`
}
Bg has parameters for a slower, low level of background inhibition based on main FFFB computed inhibition.
type Inhib ¶
type Inhib struct {
// computed feedforward inhibition
FFi float32 `desc:"computed feedforward inhibition"`
// computed feedback inhibition (total)
FBi float32 `desc:"computed feedback inhibition (total)"`
// overall value of the FFFB computed inhibition -- this is what is added into the unit Gi inhibition level (along with GiBg and any synaptic unit-driven inhibition)
Gi float32 `` /* 170-byte string literal not displayed */
// original value of the inhibition (before pool or other effects)
GiOrig float32 `desc:"original value of the inhibition (before pool or other effects)"`
// for pools, this is the layer-level inhibition that is MAX'd with the pool-level inhibition to produce the net inhibition
LayGi float32 `` /* 127-byte string literal not displayed */
// average and max Ge excitatory conductance values, which drive FF inhibition
Ge minmax.AvgMax32 `desc:"average and max Ge excitatory conductance values, which drive FF inhibition"`
// average and max Act activation values, which drive FB inhibition
Act minmax.AvgMax32 `desc:"average and max Act activation values, which drive FB inhibition"`
}
Inhib contains state values for computed FFFB inhibition
type Params ¶
type Params struct {
// enable this level of inhibition
On bool `desc:"enable this level of inhibition"`
// [def: 1.1] [viewif: On] [min: 0] [0.8-1.5 typical, can go lower or higher as needed] overall inhibition gain -- this is main parameter to adjust to change overall activation levels -- it scales both the the ff and fb factors uniformly
Gi float32 `` /* 238-byte string literal not displayed */
// [def: 1] [viewif: On] [min: 0] overall inhibitory contribution from feedforward inhibition -- multiplies average Ge (i.e., synaptic drive into layer) -- this anticipates upcoming changes in excitation, but if set too high, it can make activity slow to emerge -- see also ff0 for a zero-point for this value
FF float32 `` /* 310-byte string literal not displayed */
// [def: 1] [viewif: On] [min: 0] overall inhibitory contribution from feedback inhibition -- multiplies average activation -- this reacts to layer activation levels and works more like a thermostat (turning up when the 'heat' in the layer is too high)
FB float32 `` /* 253-byte string literal not displayed */
// [def: 1.4,3,5] [viewif: On] [min: 0] time constant in cycles, which should be milliseconds typically (tau is roughly how long it takes for value to change significantly -- 1.4x the half-life) for integrating feedback inhibitory values -- prevents oscillations that otherwise occur -- the fast default of 1.4 should be used for most cases but sometimes a slower value (3 or higher) can be more robust, especially when inhibition is strong or inputs are more rapidly changing
FBTau float32 `` /* 477-byte string literal not displayed */
// [def: 0,0.5,1] [viewif: On] what proportion of the maximum vs. average Ge to use in the feedforward inhibition computation -- 0 = all average, 1 = all max, and values in between = proportional mix between average and max (ff_netin = avg + ff_max_vs_avg * (max - avg)) -- including more max can be beneficial especially in situations where the average can vary significantly but the activity should not -- max is more robust in many situations but less flexible and sensitive to the overall distribution -- max is better for cases more closely approximating single or strictly fixed winner-take-all behavior -- 0.5 is a good compromise in many cases and generally requires a reduction of .1 or slightly more (up to .3-.5) from the gi value for 0
MaxVsAvg float32 `` /* 749-byte string literal not displayed */
// [def: 0.1] [viewif: On] feedforward zero point for average Ge -- below this level, no FF inhibition is computed based on avg Ge, and this value is subtraced from the ff inhib contribution above this value -- the 0.1 default should be good for most cases (and helps FF_FB produce k-winner-take-all dynamics), but if average Ges are lower than typical, you may need to lower it
FF0 float32 `` /* 380-byte string literal not displayed */
// [def: 0,0.05] [viewif: On] extra feedforward inhibition applied when average Ge exceeds a higher threshold -- produces a nonlinear inhibition effect that is consistent with a wide range of neuroscience data, including popout and the Reynolds & Heeger, 2009 attention model
FFEx float32 `` /* 277-byte string literal not displayed */
// [def: 0.15] [viewif: On] point of average Ge at which extra inhibition based on feedforward level starts
FFEx0 float32 `viewif:"On" def:"0.15" desc:"point of average Ge at which extra inhibition based on feedforward level starts"`
// [view: -] rate = 1 / tau
FBDt float32 `inactive:"+" view:"-" json:"-" xml:"-" desc:"rate = 1 / tau"`
}
Params parameterizes feedforward (FF) and feedback (FB) inhibition (FFFB) based on average (or maximum) Ge (FF) and activation (FB)
func (*Params) FBInhib ¶
FBInhib computes feedback inhibition value as function of average activation
func (*Params) FFInhib ¶
FFInhib returns the feedforward inhibition value based on average and max excitatory conductance within relevant scope
Source Files