fsfffb

package
v2.0.0-dev0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 16, 2024 License: BSD-3-Clause Imports: 4 Imported by: 0

README

FS (Fast & Slow) FFFB Inhibition

FFFB is the feedforward (FF) and feedback (FB) inhibition mechanism, originally developed for the Leabra model. It applies inhibition as a function of the average Ge (excitatory conductance) of units in the layer (this is FF = reflecting all the excitatory input to a layer) and average Act rate-code-like activation within the layer (which is FB = reflecting the activity level within the layer itself). Act is slowly integrated over time as a function of the ISI (inter-spike-interval).

FS-FFFB is a fast and slow (FS) version of FFFB that works directly with spike input signals instead of using "internal" variables like Ge and Act, and uses time dynamics based on an emerging consensus about the differences between three major classes of inhibitory interneurons, and their functional properties, e.g., Cardin, 2018.

  • PV: fast-spiking basket cells that target the cell bodies of excitatory neurons and coexpress the calcium-binding protein parvalbumin (PV). These are the "first responders", and are also rapidly depressing -- they provide quick control of activity, responding to FF new input and FB drive, allowing the first spiking pyramidal neurons to quickly shut off other competitors, and maintain a sparse overall level of activity. The PV activity level (and consequent inhibitory conductance into pyramidal cells, Gi) closely tracks the incoming excitatory conductance Ge, which keeps neurons in their sensitive dynamic range e.g., Shadlen & Newsome, 1994.

  • SST: more slowly responding, higher-threshold spiking cells that target the distal dendrites of excitatory neurons and coexpress the peptide somatostatin (SST). These require repetitive, facilitating, afferent input to be activated, and may regulate the dendritic integration of synaptic inputs over a longer timescale. The dependence in the original FFFB of FB on the slower integrated Act variable, which only comes on after the first spike (in order to compute the ISI), is consistent with these slower SST dynamics.

  • VIP: sparse dendrite-targeting cells that synapse onto SST interneurons and the dendrites of pyramidal neurons, and coexpress vasoactive intestinal peptide (VIP). VIP interneurons are a subset of the larger 5HT3aR-expressing interneuron class. These can provide disinhibition of SST inhibition. These are targeted by thalamic pathways into layer 1 of cortex, and may be responsible for arousal and gating-like modulation from the thalamus. We do not directly implement them in axon, but do indirectly capture their effects in the gating dynamics of the pcore model.

Remarkably, the parameters that work to enable successful error-driven learning in axon using the new FS-FFFB equations end up very closely reproducing the net inhibition produced by the original FFFB model, as shown in the following figure.

FS-FFFB vs original FFFB

Figure 1: Comparison of original FFFB inhibition (OGi) vs. new FS-FFFB inhibition (SGi) from the inhib example simulation, showing that the FS-FFFB parameters that enable successful learning produce nearly identical overall levels of inhibition compared to the original. Act.Avg shows the time-averaged activity in the lower layer that feeds into the one shown, and resembles the slow SST response, while the jagged ups and downs are due to the fast PV component.

PV is fast and highly chaotic

The dramatic swings in Gi levels as shown in the above figure are all due to the PV Fast component, which increases directly as a function of incoming FF and FB spikes, and decays with a fast time constant of 6 msec (default):

	FSi += (FFs + FB*FBs) - FSi/FSTau

where:

  • FSi = fast PV contribution to inhibition, time-integrated.
  • FFs = normalized sum of incoming FF spikes.
  • FBs = normalized sum of incoming FB spikes from neurons in the same pool being inhibited.
  • FB = weighting factor for FB spikes, which defaults to 1 but needs to be smaller for smaller networks (0.5) and larger for larger ones (e.g., 4).
  • FSTau = time constant for decaying FSi (6 msec default).

SST is a slow time-average

The slow SST contribution slowly tracks overall spiking activity in the pool, roughly as the Act.Avg green line in the above figure, based on the following equations:

	SSi += (SSf*FBs - SSi) / SSiTau
	SSf += FBs*(1-SSf) - SSf / SSfTau

where:

  • SSi = slow SST contribution to inhibition, time-integrated.
  • SSf = synaptic facilitation component for SS, which increases as a function of spiking activity as shown in the 2nd equation.
  • FBs = normalized sum of incoming FB spikes from neurons in the same pool being inhibited.
  • SSiTau = integration time constant for SSi, which is 50 msec by default (slow).
  • SSfTau = time constant for SSf, which is 20 msec by default.

Combined Gi

The combined overall inhibitory conductance Gi is based on a thresholded version of the FSi component plus a weighted contribution of the SSi level, which tends to be very weak due to the long time integral:

    Gi = |FSi > FS0|_+ + SS * SSi

where:

  • Gi = overall inhibitory conductance.
  • FSi = fast-spiking inhibition per above.
  • SSi = slow-spiking inhibition per above.
  • FS0 = threshold for FSi, default .1 as in the original FFFB, below which it contributes 0. This factor is important for filtering out small levels of incoming spikes and produces an observed nonlinearity in the Gi response.
  • SS = multiplier for SSi, which is 30 by default: SSi is relatively weak so this needs to be a strong multiplier to get into the range of FSi.

Additional SSGi -> VmDend

In addition to the pooled Gi inhibition value described above, the slow inhibition value SSi is applied with a separate weighting factor to the dendritic membrane potential update, as an additional inhibitory component, weighted by the Act.Dend.SSGi parameter (default = 2). This is important for specifically balancing the positive feedback loop in increasing NMDA channel activation, which is voltage gated as a function of VmDend. Over the course of learning VmDend tends to increase for the most active neurons, and this additional SSGi factor balances that and prevents it from entering into a runaway positive feedback loop.

Biologically, the idea is that SSi inhibition affects the overall cellular Vm, but because it comes directly into the distal dendrites, it has an extra impact there.

References

Documentation

Overview

Package fsfffb provides Fast and Slow feedforward (FF) and feedback (FB) inhibition (FFFB) based on incoming spikes (FF) and outgoing spikes (FB).

This produces a robust, graded k-Winners-Take-All dynamic of sparse distributed representations having approximately k out of N neurons active at any time, where k is typically 10-20 percent of N.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type GiParams 

type GiParams struct {

	// enable this level of inhibition
	On slbool.Bool

	// overall inhibition gain -- this is main parameter to adjust to change overall activation levels -- it scales both the the FS and SS factors uniformly
	Gi float32 `min:"0" default:"1,1.1,0.75,0.9"`

	// amount of FB spikes included in FF for driving FS -- for small networks, 0.5 or 1 works best; larger networks and more demanding inhibition requires higher levels.
	FB float32 `min:"0" default:"0.5,1,4"`

	// fast spiking (PV+) intgration time constant in cycles (msec) -- tau is roughly how long it takes for value to change significantly -- 1.4x the half-life.
	FSTau float32 `min:"0" default:"6"`

	// multiplier on SS slow-spiking (SST+) in contributing to the overall Gi inhibition -- FS contributes at a factor of 1
	SS float32 `min:"0" default:"30"`

	// slow-spiking (SST+) facilitation decay time constant in cycles (msec) -- facilication factor SSf determines impact of FB spikes as a function of spike input-- tau is roughly how long it takes for value to change significantly -- 1.4x the half-life.
	SSfTau float32 `min:"0" default:"20"`

	// slow-spiking (SST+) intgration time constant in cycles (msec) cascaded on top of FSTau -- tau is roughly how long it takes for value to change significantly -- 1.4x the half-life.
	SSiTau float32 `min:"0" default:"50"`

	// fast spiking zero point -- below this level, no FS inhibition is computed, and this value is subtracted from the FSi
	FS0 float32 `default:"0.1"`

	// time constant for updating a running average of the feedforward inhibition over a longer time scale, for computing FFPrv
	FFAvgTau float32 `default:"50"`

	// proportion of previous average feed-forward inhibition (FFAvgPrv) to add, resulting in an accentuated temporal-derivative dynamic where neurons respond most strongly to increases in excitation that exceeds inhibition from last time.
	FFPrv float32 `default:"0"`

	// minimum GeExt value required to drive external clamping dynamics (if clamp is set), where only GeExt drives inhibition.  If GeExt is below this value, then the usual FS-FFFB drivers are used.
	ClampExtMin float32 `default:"0.05"`

	// rate = 1 / tau
	FSDt float32 `edit:"-" display:"-" json:"-" xml:"-"`

	// rate = 1 / tau
	SSfDt float32 `edit:"-" display:"-" json:"-" xml:"-"`

	// rate = 1 / tau
	SSiDt float32 `edit:"-" display:"-" json:"-" xml:"-"`

	// rate = 1 / tau
	FFAvgDt float32 `edit:"-" display:"-" json:"-" xml:"-"`
	// contains filtered or unexported fields
}

GiParams parameterizes feedforward (FF) and feedback (FB) inhibition (FFFB) based on incoming spikes (FF) and outgoing spikes (FB) across Fast (PV+) and Slow (SST+) timescales. FF -> PV -> FS fast spikes, FB -> SST -> SS slow spikes (slow to get going)

func (*GiParams) Defaults 

func (fb *GiParams) Defaults()

func (*GiParams) FS 

func (fb *GiParams) FS(fsi, gext float32, clamped bool) float32

FS returns the current effective FS value based on fsi and fsd if clamped, then only use gext, without applying FS0

func (*GiParams) FS0Thr 

func (fb *GiParams) FS0Thr(val float32) float32

FS0Thr applies FS0 threshold to given value

func (*GiParams) FSiFromFFs 

func (fb *GiParams) FSiFromFFs(fsi *float32, ffs, fbs float32)

FSiFromFFs updates fast-spiking inhibition from FFs spikes

func (*GiParams) Inhib 

func (fb *GiParams) Inhib(inh *Inhib, gimult float32)

Inhib is full inhibition computation for given inhib state which has aggregated FFs and FBs spiking values

func (*GiParams) SSFromFBs 

func (fb *GiParams) SSFromFBs(ssf, ssi *float32, fbs float32)

SSFromFBs updates slow-spiking inhibition from FBs

func (*GiParams) ShouldDisplay 

func (fb *GiParams) ShouldDisplay(field string) bool

func (*GiParams) Update 

func (fb *GiParams) Update()

type Inhib 

type Inhib struct {

	// all feedforward incoming spikes into neurons in this pool -- raw aggregation
	FFsRaw float32

	// all feedback outgoing spikes generated from neurons in this pool -- raw aggregation
	FBsRaw float32

	// all extra GeExt conductances added to neurons
	GeExtRaw float32

	// all feedforward incoming spikes into neurons in this pool, normalized by pool size
	FFs float32

	// all feedback outgoing spikes generated from neurons in this pool, normalized by pool size
	FBs float32

	// all extra GeExt conductances added to neurons, normalized by pool size
	GeExts float32

	// if true, this layer is hard-clamped and should use GeExts exclusively for PV
	Clamped slbool.Bool

	// fast spiking PV+ fast integration of FFs feedforward spikes
	FSi float32

	// slow spiking SST+ integration of FBs feedback spikes
	SSi float32

	// slow spiking facilitation factor, representing facilitating effects of recent activity
	SSf float32

	// overall fast-spiking inhibitory conductance
	FSGi float32

	// overall slow-spiking inhibitory conductance
	SSGi float32

	// overall inhibitory conductance = FSGi + SSGi
	Gi float32

	// original value of the inhibition (before pool or other effects)
	GiOrig float32

	// for pools, this is the layer-level inhibition that is MAX'd with the pool-level inhibition to produce the net inhibition
	LayGi float32

	// longer time scale running average FF drive -- used for FFAvgPrv
	FFAvg float32

	// previous theta cycle FFAvg value -- for FFPrv factor -- updated in Decay function that is called at start of new ThetaCycle
	FFAvgPrv float32

	// int32 atomic add compatible integration of FFsRaw
	FFsRawInt int32

	// int32 atomic add compatible integration of FBsRaw
	FBsRawInt int32

	// int32 atomic add compatible integration of GeExtRaw
	GeExtRawInt int32
}

Inhib contains state values for computed FFFB inhibition

func (*Inhib) Decay 

func (fi *Inhib) Decay(decay float32)

Decay reduces inhibition values by given decay proportion

func (*Inhib) FloatFromInt 

func (fi *Inhib) FloatFromInt(ival int32) float32

FloatFromInt converts the given int32 value produced via FloatToInt back into a float32 (divides by factor)

func (*Inhib) FloatFromIntFactor 

func (fi *Inhib) FloatFromIntFactor() float32

FloatFromIntFactor returns the factor used for converting int32 back to float32 -- this is 1 / FloatToIntFactor for faster multiplication instead of dividing.

func (*Inhib) FloatToInt 

func (fi *Inhib) FloatToInt(val float32, nneurons int) int32

FloatToInt converts the given floating point value to a large int for max updating.

func (*Inhib) FloatToIntFactor 

func (fi *Inhib) FloatToIntFactor() float32

FloatToIntFactor returns the factor used for converting float32 to int32 for summing, assuming that the overall value is in the general order of 0-1 (512 is the max).

func (*Inhib) GiFromFSSS 

func (fi *Inhib) GiFromFSSS() float32

GiFromFSSS returns the sum of FSGi and SSGi as overall inhibition

func (*Inhib) Init 

func (fi *Inhib) Init()

func (*Inhib) InitRaw 

func (fi *Inhib) InitRaw()

InitRaw clears raw spike counters -- done every cycle prior to accumulating

func (*Inhib) IntToRaw 

func (fi *Inhib) IntToRaw()

IntToRaw computes int values into float32 raw values

func (*Inhib) LayerMax 

func (fi *Inhib) LayerMax(liGi float32)

LayerMax updates given pool-level inhib values from given layer-level Gi with resulting value being the Max of either

func (*Inhib) PoolMax 

func (fi *Inhib) PoolMax(piGi float32)

PoolMax updates given layer-level inhib values from given pool-level with resulting value being the Max of either

func (*Inhib) RawIncr 

func (fi *Inhib) RawIncr(spike, geRaw, geExt float32, nneurons int)

RawIncr increments raw values from given neuron-based input values

func (*Inhib) RawIncrInt 

func (fi *Inhib) RawIncrInt(spike, geRaw, geExt float32, nneurons int)

RawIncrInt increments raw values from given neuron-based input values for the int-based values (typically use Atomic InterlockedAdd instead)

func (*Inhib) SaveOrig 

func (fi *Inhib) SaveOrig()

SaveOrig saves the current Gi values as original values

func (*Inhib) SpikesFromRaw 

func (fi *Inhib) SpikesFromRaw(nneurons int)

SpikesFromRaw updates spike values from raw, dividing by given number in pool

func (*Inhib) Zero 

func (fi *Inhib) Zero()

Zero resets all accumulating inhibition factors to 0

type Inhibs 

type Inhibs []Inhib

Inhibs is a slice of Inhib records

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.