util

Documentation

Index

• Variables
• func ArgMax(s []float32) (int, float32, error)
• func CloseFile(file io.Closer) error
• func GetPathType(path string) string
• func Mean(vector []float32) float32
• func Norm(v []float32, p int) float64
• func Normalize(embedding []float32, p int) []float32
• func OpenFile(filename string) (io.ReadCloser, error)
• func PathJoinSafe(elem ...string) string
• func ReadFileBytes(filename string) ([]byte, error)
• func ReadLine(r *bufio.Reader) ([]byte, error)
• func Sigmoid(s []float32) []float32
• func SoftMax(vector []float32) []float32
• func SumSlice(s []float64) float64

Variables

var FileSystem = afs.New()

Functions

func ArgMax

func ArgMax(s []float32) (int, float32, error)

ArgMax find both index of max value in s and max value.

func CloseFile

func CloseFile(file io.Closer) error

func GetPathType

func GetPathType(path string) string

func Mean

func Mean(vector []float32) float32

Mean of a float32 vector.

func Norm

func Norm(v []float32, p int) float64

Norm of a vector.

func Normalize

func Normalize(embedding []float32, p int) []float32

Normalize single vector according to: https://pytorch.org/docs/stable/generated/torch.nn.functional.normalize.html

func OpenFile

func OpenFile(filename string) (io.ReadCloser, error)

func PathJoinSafe

func PathJoinSafe(elem ...string) string

PathJoinSafe wrapper around filepath.Join to ensure that paths are correctly constructed if the path is a normal OS path, just use filepath.Join if the path is S3, trim any trailing slashes and construct it manually from the components so that double slashes (e.g. s3://) are preserved.

func ReadFileBytes

func ReadFileBytes(filename string) ([]byte, error)

func ReadLine

func ReadLine(r *bufio.Reader) ([]byte, error)

ReadLine returns a single line (without the ending \n) from the input buffered reader. An error is returned if there is an error with the buffered reader. This function is needed to avoid the 65K char line limit

func Sigmoid

func Sigmoid(s []float32) []float32

func SoftMax

func SoftMax(vector []float32) []float32

SoftMax take a vector and calculate softmax scores of its values.

func SumSlice

func SumSlice(s []float64) float64