fsfffb

README

FS (Fast & Slow) FFFB Inhibition

FFFB is the feedforward (FF) and feedback (FB) inhibition mechanism, originally developed for the Leabra model. It applies inhibition as a function of the average Ge (excitatory conductance) of units in the layer (this is FF = reflecting all the excitatory input to a layer) and average Act rate-code-like activation within the layer (which is FB = reflecting the activity level within the layer itself). Act is slowly integrated over time as a function of the ISI (inter-spike-interval).

FS-FFFB is a fast and slow (FS) version of FFFB that works directly with spike input signals instead of using "internal" variables like Ge and Act, and uses time dynamics based on an emerging consensus about the differences between three major classes of inhibitory interneurons, and their functional properties, e.g., Cardin, 2018.

• PV: fast-spiking basket cells that target the cell bodies of excitatory neurons and coexpress the calcium-binding protein parvalbumin (PV). These are the "first responders", and are also rapidly depressing -- they provide quick control of activity, responding to FF new input and FB drive, allowing the first spiking pyramidal neurons to quickly shut off other competitors, and maintain a sparse overall level of activity. The PV activity level (and consequent inhibitory conductance into pyramidal cells, Gi) closely tracks the incoming excitatory conductance Ge, which keeps neurons in their sensitive dynamic range e.g., Shadlen & Newsome, 1994.

• SST: more slowly-responding, higher-threshold spiking cells that target the distal dendrites of excitatory neurons and coexpress the peptide somatostatin (SST). These require repetitive, facilitating, afferent input to be activated, and may regulate the dendritic integration of synaptic inputs over a longer timescale. The dependence in the original FFFB of FB on the slower integrated Act variable, which only comes on after the first spike (in order to compute the ISI), is consistent with these slower SST dynamics.

• VIP: sparse dendrite-targeting cells that synapse onto SST interneurons and the dendrites of pyramidal neurons, and coexpress vasoactive intestinal peptide (VIP). VIP interneurons are a subset of the larger 5HT3aR-expressing interneuron class. These can provide disinhibition of SST inhibition. These are targeted by thalamic projections into layer 1 of cortex, and may be responsible for arousal and gating-like modulation from the thalamus. We do not directly implement them in axon, but do indirectly capture their effects in the gating dynamics of the pcore model.

Remarkably, the parameters that work to enable successful error-driven learning in axon using the new FS-FFFB equations end up very closely reproducing the net inhibition produced by the original FFFB model, as shown in the following figure.

Figure 1: Comparison of original FFFB inhibition (OGi) vs. new FS-FFFB inhibition (SGi) from the inhib example simulation, showing that the FS-FFFB parameters that enable successful learning produce nearly identical overall levels of inhibition compared to the original. Act.Avg shows the time-averaged activity in the lower layer that feeds into the one shown, and resembles the slow SST response, while the jagged ups and downs are due to the fast PV component.

PV is fast and highly chaotic

The dramatic swings in Gi levels as shown in the above figure are all due to the PV Fast component, which increases directly as a function of incoming FF and FB spikes, and decays with a fast time constant of 6 msec (default):

	FSi += (FFs + FB*FBs) - FSi/FSTau

where:

• FSi = fast PV contribution to inhibition, time-integrated.
• FFs = normalized sum of incoming FF spikes.
• FBs = normalized sum of incoming FB spikes from neurons in the same pool being inhibited.
• FB = weighting factor for FB spikes, which defaults to 1 but needs to be smaller for smaller networks (0.5) and larger for larger ones (e.g., 4).
• FSTau = time constant for decaying FSi (6 msec default).

SST is a slow time-average

The slow SST contribution slowly tracks overall spiking activity in the pool, roughly as the Act.Avg green line in the above figure, based on the following equations:

	SSi += (SSf*FBs - SSi) / SSiTau
	SSf += FBs*(1-SSf) - SSf / SSfTau

where:

• SSi = slow SST contribution to inhibition, time-integrated.
• SSf = synaptic facilitation component for SS, which increases as a function of spiking activity as shown in the 2nd equation.
• FBs = normalized sum of incoming FB spikes from neurons in the same pool being inhibited.
• SSiTau = integration time constant for SSi, which is 50 msec by default (slow).
• SSfTau = time constant for SSf, which is 20 msec by default.

Combined Gi

The combined overall inhibitory conductance Gi is based on a thresholded version of the FSi component plus a weighted contribution of the SSi level, which tends to be very weak due to the long time integral:

    Gi = |FSi > FS0|_+ + SS * SSi

where:

• Gi = overall inhibitory conductance.
• FSi = fast-spiking inhibition per above.
• SSi = slow-spiking inhibition per above.
• FS0 = threshold for FSi, default .1 as in the original FFFB, below which it contributes 0. This factor is important for filtering out small levels of incoming spikes and produces an observed nonlinearity in the Gi response.
• SS = multiplier for SSi, which is 30 by default: SSi is relatively weak so this needs to be a strong multiplier to get into the range of FSi.

Additional SSGi -> VmDend

In addition to the pooled Gi inhibition value described above, the slow inhibition value SSi is applied with a separate weighting factor to the dendritic membrane potential update, as an additional inhibitory component, weighted by the Act.Dend.SSGi parameter (default = 2). This is important for specifically balancing the positive feedback loop in increasing NMDA channel activation, which is voltage gated as a function of VmDend. Over the course of learning VmDend tends to increase for the most active neurons, and this additional SSGi factor balances that and prevents it from entering into a runaway positive feedback loop.

Biologically, the idea is that SSi inhibition affects the overall cellular Vm, but because it comes directly into the distal dendrites, it has an extra impact there.

References

• Cardin, J. A. (2018). Inhibitory interneurons regulate temporal precision and correlations in cortical circuits. Trends in Neurosciences, 41(10), 689–700. https://doi.org/10.1016/j.tins.2018.07.015

• Shadlen, M. N., & Newsome, W. T. (1994). Noise, neural codes and cortical organization. Current Opinion in Neurobiology, 4, 569–579. http://www.ncbi.nlm.nih.gov/pubmed/7812147

Documentation

Overview

Package fsfffb provides Fast and Slow feedforward (FF) and feedback (FB) inhibition (FFFB) based on incoming spikes (FF) and outgoing spikes (FB).

This produces a robust, graded k-Winners-Take-All dynamic of sparse distributed representations having approximately k out of N neurons active at any time, where k is typically 10-20 percent of N.

Index

• type GiParams
  • func (fb *GiParams) Defaults()
  • func (fb *GiParams) FS(fsi, gext float32, clamped bool) float32
  • func (fb *GiParams) FS0Thr(val float32) float32
  • func (fb *GiParams) FSiFmFFs(fsi *float32, ffs, fbs float32)
  • func (fb *GiParams) Inhib(inh *Inhib, gimult float32)
  • func (fb *GiParams) SSFmFBs(ssf, ssi *float32, fbs float32)
  • func (fb *GiParams) Update()
• type Inhib
  • func (fi *Inhib) Decay(decay float32)
  • func (fi *Inhib) FloatFmIntFactor() float32
  • func (fi *Inhib) FloatFromInt(ival int32) float32
  • func (fi *Inhib) FloatToInt(val float32) int32
  • func (fi *Inhib) FloatToIntFactor() float32
  • func (fi *Inhib) GiFmFSSS() float32
  • func (fi *Inhib) Init()
  • func (fi *Inhib) InitRaw()
  • func (fi *Inhib) IntToRaw()
  • func (fi *Inhib) LayerMax(liGi float32)
  • func (fi *Inhib) PoolMax(piGi float32)
  • func (fi *Inhib) RawIncr(spike, geRaw, geExt float32)
  • func (fi *Inhib) RawIncrInt(spike, geRaw, geExt float32)
  • func (fi *Inhib) SaveOrig()
  • func (fi *Inhib) SpikesFmRaw(npool int)
  • func (fi *Inhib) Zero()
• type Inhibs

Types

type GiParams

type GiParams struct {
	On       slbool.Bool `desc:"enable this level of inhibition"`
	Gi       float32     `` /* 249-byte string literal not displayed */
	FB       float32     `` /* 204-byte string literal not displayed */
	FSTau    float32     `` /* 188-byte string literal not displayed */
	SS       float32     `` /* 152-byte string literal not displayed */
	SSfTau   float32     `` /* 284-byte string literal not displayed */
	SSiTau   float32     `` /* 215-byte string literal not displayed */
	FS0      float32     `` /* 145-byte string literal not displayed */
	FFAvgTau float32     `` /* 148-byte string literal not displayed */
	FFPrv    float32     `` /* 259-byte string literal not displayed */

	FSDt    float32 `inactive:"+" view:"-" json:"-" xml:"-" desc:"rate = 1 / tau"`
	SSfDt   float32 `inactive:"+" view:"-" json:"-" xml:"-" desc:"rate = 1 / tau"`
	SSiDt   float32 `inactive:"+" view:"-" json:"-" xml:"-" desc:"rate = 1 / tau"`
	FFAvgDt float32 `inactive:"+" view:"-" json:"-" xml:"-" desc:"rate = 1 / tau"`
	// contains filtered or unexported fields
}

GiParams parameterizes feedforward (FF) and feedback (FB) inhibition (FFFB) based on incoming spikes (FF) and outgoing spikes (FB) across Fast (PV+) and Slow (SST+) timescales. FF -> PV -> FS fast spikes, FB -> SST -> SS slow spikes (slow to get going)

func (*GiParams) Defaults

func (fb *GiParams) Defaults()

func (*GiParams) FS

func (fb *GiParams) FS(fsi, gext float32, clamped bool) float32

FS returns the current effective FS value based on fsi and fsd if clamped, then only use gext, without applying FS0

func (*GiParams) FS0Thr

func (fb *GiParams) FS0Thr(val float32) float32

FS0Thr applies FS0 threshold to given value

func (*GiParams) FSiFmFFs

func (fb *GiParams) FSiFmFFs(fsi *float32, ffs, fbs float32)

FSiFmFFs updates fast-spiking inhibition from FFs spikes

func (*GiParams) Inhib

func (fb *GiParams) Inhib(inh *Inhib, gimult float32)

Inhib is full inhibition computation for given inhib state which has aggregated FFs and FBs spiking values

func (*GiParams) SSFmFBs

func (fb *GiParams) SSFmFBs(ssf, ssi *float32, fbs float32)

SSFmFBs updates slow-spiking inhibition from FBs

func (*GiParams) Update

func (fb *GiParams) Update()

type Inhib

type Inhib struct {
	FFsRaw   float32     `desc:"all feedforward incoming spikes into neurons in this pool -- raw aggregation"`
	FBsRaw   float32     `desc:"all feedback outgoing spikes generated from neurons in this pool -- raw aggregation"`
	GeExtRaw float32     `desc:"all extra GeExt conductances added to neurons"`
	FFs      float32     `desc:"all feedforward incoming spikes into neurons in this pool, normalized by pool size"`
	FBs      float32     `desc:"all feedback outgoing spikes generated from neurons in this pool, normalized by pool size"`
	GeExts   float32     `desc:"all extra GeExt conductances added to neurons, normalized by pool size"`
	Clamped  slbool.Bool `desc:"if true, this layer is hard-clamped and should use GeExts exclusively for PV"`
	FSi      float32     `desc:"fast spiking PV+ fast integration of FFs feedforward spikes"`
	SSi      float32     `desc:"slow spiking SST+ integration of FBs feedback spikes"`
	SSf      float32     `desc:"slow spiking facilitation factor, representing facilitating effects of recent activity"`
	FSGi     float32     `desc:"overall fast-spiking inhibitory conductance"`
	SSGi     float32     `desc:"overall slow-spiking inhibitory conductance"`
	Gi       float32     `desc:"overall inhibitory conductance = FSGi + SSGi"`
	GiOrig   float32     `desc:"original value of the inhibition (before pool or other effects)"`
	LayGi    float32     `` /* 127-byte string literal not displayed */
	FFAvg    float32     `desc:"longer time scale running average FF drive -- used for FFAvgPrv"`
	FFAvgPrv float32     `` /* 130-byte string literal not displayed */

	FFsRawInt   int32 `desc:"int32 atomic add compatible integration of FFsRaw"`
	FBsRawInt   int32 `desc:"int32 atomic add compatible integration of FBsRaw"`
	GeExtRawInt int32 `desc:"int32 atomic add compatible integration of GeExtRaw"`
}

Inhib contains state values for computed FFFB inhibition

func (*Inhib) Decay

func (fi *Inhib) Decay(decay float32)

Decay reduces inhibition values by given decay proportion

func (*Inhib) FloatFmIntFactor

func (fi *Inhib) FloatFmIntFactor() float32

FloatFmIntFactor returns the factor used for converting int32 back to float32 -- this is 1 / FloatToIntFactor for faster multiplication instead of dividing.

func (*Inhib) FloatFromInt

func (fi *Inhib) FloatFromInt(ival int32) float32

FloatFromInt converts the given int32 value produced via FloatToInt back into a float32 (divides by factor)

func (*Inhib) FloatToInt

func (fi *Inhib) FloatToInt(val float32) int32

FloatToInt converts the given floating point value to a large int for max updating.

func (*Inhib) FloatToIntFactor

func (fi *Inhib) FloatToIntFactor() float32

FloatToIntFactor returns the factor used for converting float32 to int32 for summing, assuming that the overall value is in the general order of 0-1 (512 is the max).

func (*Inhib) GiFmFSSS

func (fi *Inhib) GiFmFSSS() float32

GiFmFSSS returns the sum of FSGi and SSGi as overall inhibition

func (*Inhib) Init

func (fi *Inhib) Init()

func (*Inhib) InitRaw

func (fi *Inhib) InitRaw()

InitRaw clears raw spike counters -- done every cycle prior to accumulating

func (*Inhib) IntToRaw

func (fi *Inhib) IntToRaw()

IntToRaw computes int values into float32 raw values

func (*Inhib) LayerMax

func (fi *Inhib) LayerMax(liGi float32)

LayerMax updates given pool-level inhib values from given layer-level Gi with resulting value being the Max of either

func (*Inhib) PoolMax

func (fi *Inhib) PoolMax(piGi float32)

PoolMax updates given layer-level inhib values from given pool-level with resulting value being the Max of either

func (*Inhib) RawIncr

func (fi *Inhib) RawIncr(spike, geRaw, geExt float32)

RawIncr increments raw values from given neuron-based input values

func (*Inhib) RawIncrInt

func (fi *Inhib) RawIncrInt(spike, geRaw, geExt float32)

RawIncrInt increments raw values from given neuron-based input values for the int-based values (typically use Atomic InterlockedAdd instead)

func (*Inhib) SaveOrig

func (fi *Inhib) SaveOrig()

SaveOrig saves the current Gi values as original values

func (*Inhib) SpikesFmRaw

func (fi *Inhib) SpikesFmRaw(npool int)

SpikesFmRaw updates spike values from raw, dividing by given number in pool

func (*Inhib) Zero

func (fi *Inhib) Zero()

Zero resets all accumulating inhibition factors to 0

type Inhibs

type Inhibs []Inhib

Inhibs is a slice of Inhib records