README
¶
vision
This repository contains visual processing packages in Go (golang), focused mainly on providing efficient V1 (primary visual cortex) level filtering of images, with the output then suitable as input for neural networks.
Two main types of filters are supported:
-
Gabor filters simulate V1 simple-cell responses in terms of an oriented sine wave times a gaussian envelope that localizes the filter in space. This produces an edge detector that detects oriented contrast transitions between light and dark. In general, the main principle of primary visual filtering is to focus on spatial (and temporal) changes, while filtering out static, uniform areas.
-
DoG (difference of gaussian) filters simulate retinal On-center vs. Off-center contrast coding cells -- unlike gabor filters, these do not have orientation tuning. Mathematically, they are a difference between a narrow (center) vs wide (surround) gaussian, of opposite signs, balanced so that a uniform input generates offsetting values that sum to zero. In the visual system, orientation tuning is constructed from aligned DoG-like inputs, but it is more efficient to just use the Gabor filters directly. However, DoG filters capture the "blob" cells that encode color contrasts.
The vfilter package contains general-purpose filtering code that applies (convolves) any given filter with a visual input. It also supports converting an image.Image into a tensor.Float32 tensor which is the main data type used in this framework. It also supports max-pooling for efficiently reducing the dimensionality of inputs.
The kwta package provides an implementation of the feedforward and feedback (FFFB) inhibition dynamics (and noisy X-over-X-plus-1 activation function) from the Leabra algorithm to produce a k-Winners-Take-All processing of visual filter outputs -- this increases the contrast and simplifies the representations, and is a good model of the dynamics in primary visual cortex.
Documentation
¶
Overview ¶
Package vision is a repository containing visual processing packages in Go (golang), focused mainly on providing efficient V1 (primary visual cortex) level filtering of images, with the output then suitable as input for neural networks.
Two main types of filters are supported:
* **Gabor** filters simulate V1 simple-cell responses in terms of an oriented sine wave times a gaussian envelope that localizes the filter in space. This produces an edge detector that detects oriented contrast transitions between light and dark. In general, the main principle of primary visual filtering is to focus on spatial (and temporal) changes, while filtering out static, uniform areas.
* **DoG** (difference of gaussian) filters simulate retinal On-center vs. Off-center contrast coding cells -- unlike gabor filters, these do not have orientation tuning. Mathematically, they are a difference between a narrow (center) vs wide (surround) gaussian, of opposite signs, balanced so that a uniform input generates offsetting values that sum to zero. In the visual system, orientation tuning is constructed from aligned DoG-like inputs, but it is more efficient to just use the Gabor filters directly. However, DoG filters capture the "blob" cells that encode color contrasts.
The `vfilter` package contains general-purpose filtering code that applies (convolves) any given filter with a visual input. It also supports converting an `image.Image` into a `tensor.Float32` tensor which is the main data type used in this framework. It also supports max-pooling for efficiently reducing the dimensionality of inputs.
The `kwta` package provides an implementation of the feedforward and feedback (FFFB) inhibition dynamics (and noisy X-over-X-plus-1 activation function) from the `Leabra` algorithm to produce a k-Winners-Take-All processing of visual filter outputs -- this increases the contrast and simplifies the representations, and is a good model of the dynamics in primary visual cortex.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
package dog provides the Difference-of-Gaussians (DoG) filter for visual and other forms of signal processing
|
package dog provides the Difference-of-Gaussians (DoG) filter for visual and other forms of signal processing |
|
examples
|
|
|
Package fffb provides feedforward (FF) and feedback (FB) inhibition (FFFB) based on average (or maximum) excitatory netinput (FF) and activation (FB).
|
Package fffb provides feedforward (FF) and feedback (FB) inhibition (FFFB) based on average (or maximum) excitatory netinput (FF) and activation (FB). |
|
package gabor provides a gabor filter for visual and other forms of signal processing
|
package gabor provides a gabor filter for visual and other forms of signal processing |
|
package kwta provides the Leabra k-Winners-Take-All inhibition function in a form that can be applied to an tensor of float32 values as computed by visual (or other modality) filtering routines.
|
package kwta provides the Leabra k-Winners-Take-All inhibition function in a form that can be applied to an tensor of float32 values as computed by visual (or other modality) filtering routines. |
|
Package nproc provides number of processors using slurm env var SLURM_CPUS_PER_TASK or runtime.NumCPU().
|
Package nproc provides number of processors using slurm env var SLURM_CPUS_PER_TASK or runtime.NumCPU(). |
|
Package nxx1 provides the Noisy-X-over-X-plus-1 activation function that well-characterizes the neural response function empirically, as a saturating sigmoid-like nonlinear response with an initial largely linear regime.
|
Package nxx1 provides the Noisy-X-over-X-plus-1 activation function that well-characterizes the neural response function empirically, as a saturating sigmoid-like nonlinear response with an initial largely linear regime. |
|
Package v1complex implements V1 (primary visual cortex) complex-cell filters, which operate on the output of V1 simple-cell (gabor) filters.
|
Package v1complex implements V1 (primary visual cortex) complex-cell filters, which operate on the output of V1 simple-cell (gabor) filters. |
|
Package vfilter provides filtering methods for the vision package.
|
Package vfilter provides filtering methods for the vision package. |
|
Package vxform supports visual (image) transformations: translation, scaling, rotation.
|
Package vxform supports visual (image) transformations: translation, scaling, rotation. |