toygraph

package module
v0.0.0-...-0a601b1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 10, 2024 License: MIT Imports: 4 Imported by: 0

README

ToyGraph - A Go Project for Computational Graphs

I created ToyGraph as a hands-on way to better understand computational graphs and automatic differentiation. While I’m not making any speed guarantees, I’ve focused on performance where I can (e.g., reusing allocated matrices).

ToyGraph is designed to be flexible with the types of data it can handle. I started with single-value variables and gradually expanded to include matrices and vectors, all while keeping support for scalar values. The nodes in ToyGraph are typed, ensuring type safety at compile time, though matrix/vector shape safety is only checked at runtime. You can mix scalar and matrix/vector values within the same graph, but be aware that I don’t plan to add tensor support—so this probably isn’t the tool for machine learning.

Currently, ToyGraph only runs on the CPU, though extending it to GPU support is something I might explore down the line (it would require a lot more code and operations). While the number of operations available is still limited, adding more is pretty straightforward. You can even use your own types, like a different matrix package than gonum, to build a graph.

What can it do?

ToyGraph can perform both the forward pass and automatic differentiation on any equation. Variables in the equation can be either scalars (float64, float32, int), or gonum/mat dense vectors and matrices. It works by first getting you to build a graph (an equation represented as a set of nodes with connections between them). You can then set the values of the variable on this graph, then perform a forward pass, which computes the result of the equation (you can also see the result of any single node in the graph). If you want, you can also set the gradients of the output node, and the perform a backward pass which propagates these gradients back through the graph. You can then do various things such as gradient descent (this is the way neural nets learn). ToyGraph can have as many inputs and outputs as you want.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Get 

func Get[T any](conn Conn[T]) T

func GetGrad 

func GetGrad[T any](conn Conn[T]) T

func NewMatAllocatorLike 

func NewMatAllocatorLike(m *mat.Dense) *matAllocator

NewMatAllocatorLike creates a new Allocator which actos on matricies of the same shape as m

func NewVecAllocator 

func NewVecAllocator(n int) *vecAllocator

func NewVecAllocatorLike 

func NewVecAllocatorLike(v *mat.VecDense) *vecAllocator

NewVecAllocatorLike creates a new Allocator which actos on vectors of the same length as v

func Set 

func Set[T any](conn Conn[T], val T)

func SetGrad 

func SetGrad[T any](conn Conn[T], val T)

Types

type Allocator 

type Allocator[T any] interface {
	// Create a new pointer to the data type
	New() *T
	// Set the data type to its zero value
	Zero(*T)
	// Check if the provided pointer to a data type is valid for this allocator.
	// For example, if we are allocating vecotrs, we will check if the provided vector has the same length as the vectors the allocator creates.
	Allowed(*T) error
}

An allocator is a utility class used to manipulate the buffers for operations to act on. Each node has a unique allocator for every buffer that it creates. A value allocator can contain data about the buffer to allocate, for example how long the vectors that it allocates should be.

type BinaryNode 

type BinaryNode[T, U, V any] struct {
	// contains filtered or unexported fields
}

func MatAddElem 

func MatAddElem(g *Graph, inEpA Conn[*mat.Dense], inEpB Conn[*mat.Dense]) *BinaryNode[*mat.Dense, *mat.Dense, *mat.Dense]

MatAddElem creates a node that performs an element-wise add operation

func MatMulElem 

func MatMulElem(g *Graph, inEpA Conn[*mat.Dense], inEpB Conn[*mat.Dense]) *BinaryNode[*mat.Dense, *mat.Dense, *mat.Dense]

func MatMulVec 

func MatMulVec(g *Graph, inEpA Conn[*mat.Dense], inEpB Conn[*mat.VecDense]) *BinaryNode[*mat.Dense, *mat.VecDense, *mat.VecDense]

func NewBinaryNode 

func NewBinaryNode[T, U, V any](g *Graph, viV Allocator[V], op BinaryOp[T, U, V], aEp Conn[T], bEp Conn[U]) *BinaryNode[T, U, V]

func NumAdd 

func NumAdd[T Num](g *Graph, inEpA Conn[T], inEpB Conn[T]) *BinaryNode[T, T, T]

func NumMul 

func NumMul[T Num](g *Graph, inEpA Conn[T], inEpB Conn[T]) *BinaryNode[T, T, T]

func VecAddElem 

func VecAddElem(g *Graph, inEpA Conn[*mat.VecDense], inEpB Conn[*mat.VecDense]) *BinaryNode[*mat.VecDense, *mat.VecDense, *mat.VecDense]

func VecMulElem 

func VecMulElem(g *Graph, inEpA Conn[*mat.VecDense], inEpB Conn[*mat.VecDense]) *BinaryNode[*mat.VecDense, *mat.VecDense, *mat.VecDense]

func (*BinaryNode[T, U, V]) Backward 

func (n *BinaryNode[T, U, V]) Backward()

func (*BinaryNode[T, U, V]) C 

func (n *BinaryNode[T, U, V]) C() (val, grad *V, allowed func(*V) error)

func (*BinaryNode[T, U, V]) Forward 

func (n *BinaryNode[T, U, V]) Forward()

func (*BinaryNode[T, U, V]) ZeroGrad 

func (n *BinaryNode[T, U, V]) ZeroGrad()

type BinaryOp 

type BinaryOp[T, U, V any] interface {
	ComputeForward(aVal *T, bVal *U, resVal *V)
	ComputeGrad(aVal, aGrad *T, bVal, bGrad *U, resVal, resGrad *V)
}

type Conn 

type Conn[T any] func() (value, gradient *T, allowedSet func(*T) error)

An endpoint returns a refrence to where the vector is stored

type Graph 

type Graph struct {
	// contains filtered or unexported fields
}

func NewGraph 

func NewGraph() *Graph

func (*Graph) Add 

func (g *Graph) Add(n Node)

func (*Graph) AllBackward 

func (g *Graph) AllBackward()

func (*Graph) AllForward 

func (g *Graph) AllForward()

func (*Graph) AllZeroGrads 

func (g *Graph) AllZeroGrads()

func (*Graph) Len 

func (g *Graph) Len() int

type Node 

type Node interface {
	Forward()
	Backward()
	ZeroGrad()
}

type Num 

type Num interface {
	float64 | float32 | int
}

type UnaryNode 

type UnaryNode[T, U any] struct {
	// contains filtered or unexported fields
}

func MatCosElem 

func MatCosElem(g *Graph, inEp Conn[*mat.Dense]) *UnaryNode[*mat.Dense, *mat.Dense]

MatCosElem creates a node that performs an element-wise cos operation

func MatExpElem 

func MatExpElem(g *Graph, inEp Conn[*mat.Dense]) *UnaryNode[*mat.Dense, *mat.Dense]

MatExpElem creates a node that performs an element-wise e^x operation

func MatSinElem 

func MatSinElem(g *Graph, inEp Conn[*mat.Dense]) *UnaryNode[*mat.Dense, *mat.Dense]

MatSinElem creates a node that performs an element-wise sin operation

func NewUnaryNode 

func NewUnaryNode[T, U any](g *Graph, alloc Allocator[U], op UnaryOp[T, U], inEp Conn[T]) *UnaryNode[T, U]

func NumCos 

func NumCos[T Num](g *Graph, inEp Conn[T]) *UnaryNode[T, T]

func NumExp 

func NumExp[T Num](g *Graph, inEp Conn[T]) *UnaryNode[T, T]

func NumSin 

func NumSin[T Num](g *Graph, inEp Conn[T]) *UnaryNode[T, T]

func VecCosElem 

func VecCosElem(g *Graph, inEp Conn[*mat.VecDense]) *UnaryNode[*mat.VecDense, *mat.VecDense]

VecCosElem creates a node that performs an element-wise cos operation

func VecExpElem 

func VecExpElem(g *Graph, inEp Conn[*mat.VecDense]) *UnaryNode[*mat.VecDense, *mat.VecDense]

VecExpElem creates a node that performs an element-wise e^x operation

func VecSinElem 

func VecSinElem(g *Graph, inEp Conn[*mat.VecDense]) *UnaryNode[*mat.VecDense, *mat.VecDense]

VecSinElem creates a node that performs an element-wise sin operation

func (*UnaryNode[T, U]) Backward 

func (n *UnaryNode[T, U]) Backward()

func (*UnaryNode[T, U]) C 

func (n *UnaryNode[T, U]) C() (val, grad *U, allowed func(*U) error)

func (*UnaryNode[T, U]) Forward 

func (n *UnaryNode[T, U]) Forward()

func (*UnaryNode[T, U]) ZeroGrad 

func (n *UnaryNode[T, U]) ZeroGrad()

type UnaryOp 

type UnaryOp[T, U any] interface {
	ComputeForward(inVal *T, resVal *U)
	ComputeGrad(inVal, inGrad *T, resVal, resGrad *U)
}

type ValueNode 

type ValueNode[T any] struct {
	// contains filtered or unexported fields
}

func MatValue 

func MatValue(g *Graph, rows, cols int) *ValueNode[*mat.Dense]

MatValue creates a new ValueNode that holds a matrix of dims rows, cols

func NewValueNode 

func NewValueNode[T any](g *Graph, vi Allocator[T]) *ValueNode[T]

func NumValue 

func NumValue[T Num](g *Graph) *ValueNode[T]

*************************************************** Inputs ***************************************************

func VecValue 

func VecValue(g *Graph, size int) *ValueNode[*mat.VecDense]

VecValue creates a new ValueNode that holds a vector of length size

func (*ValueNode[T]) Backward 

func (n *ValueNode[T]) Backward()

func (*ValueNode[T]) C 

func (n *ValueNode[T]) C() (value, gradient *T, allowed func(*T) error)

func (*ValueNode[T]) Forward 

func (n *ValueNode[T]) Forward()

func (*ValueNode[T]) ZeroGrad 

func (n *ValueNode[T]) ZeroGrad()

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.