util

Documentation

Overview

Package util contains utility classes for the graph storage.

GraphError

Models a graph related error. Low-level errors should be wrapped in a GraphError before they are returned to a client.

IndexManager

Manages the full text search index. The index supports simple word searches as well as phrase searches.

The index is a basically a key-value lookup which manages 2 types of entries:

Each node attribute value is split up into words. Each word gets an entry:

PrefixAttrWord + attr num + word (string) -> ids + pos (provides word and phrase lookup)

Each node attribute value is also converted into a MD5 sum which makes attribute value lookups very efficient:

PrefixAttrHash + attr num + hash (md5) -> ids (provides exact match lookup)

NamesManager

Manages names of kinds, roles and attributes. Each stored name gets either a 16 or 32 bit (little endian) number assigned. The manager provides functions to lookup either the names or their numbers.

Index

• Constants
• Variables
• type GraphError
  • func (ge *GraphError) Error() string
• type IndexManager
  • func NewIndexManager(htree *hash.HTree) *IndexManager
  • func (im *IndexManager) Count(attr, word string) (int, error)
  • func (im *IndexManager) Deindex(key string, obj map[string]string) error
  • func (im *IndexManager) Index(key string, obj map[string]string) error
  • func (im *IndexManager) LookupPhrase(attr, phrase string) ([]string, error)
  • func (im *IndexManager) LookupValue(attr, value string) ([]string, error)
  • func (im *IndexManager) LookupWord(attr, word string) (map[string][]uint64, error)
  • func (im *IndexManager) Reindex(key string, newObj map[string]string, oldObj map[string]string) error
  • func (im *IndexManager) String() string
• type NamesManager
  • func NewNamesManager(nameDB map[string]string) *NamesManager
  • func (gs *NamesManager) Decode16(val string) string
  • func (gs *NamesManager) Decode32(val string) string
  • func (gs *NamesManager) Encode16(val string, create bool) string
  • func (gs *NamesManager) Encode32(val string, create bool) string

Constants

const MaxKeysetSize = 1000

MaxKeysetSize is the maximum number of keys for a single word lookup.

const Prefix16Bit = "\x01"

Prefix16Bit is the prefix for 16 bit kind related entries

const Prefix32Bit = "\x02"

Prefix32Bit is the prefix for attribute related entries

const PrefixAttrHash = "\x01"

PrefixAttrHash is the prefix used for hashes of attribute values

const PrefixAttrWord = "\x01"

PrefixAttrWord is the prefix used for word index entries

const PrefixCode = "\x00"

PrefixCode is the prefix for entries storing codes

const PrefixCounter = "\x00"

PrefixCounter is the prefix for counter entries

const PrefixName = "\x01"

PrefixName is the prefix for entries storing names

Variables

var (
	ErrOpening         = errors.New("Failed to open graph storage")
	ErrFlushing        = errors.New("Failed to flush changes")
	ErrRollback        = errors.New("Failed to rollback changes")
	ErrClosing         = errors.New("Failed to close graph storage")
	ErrAccessComponent = errors.New("Failed to access graph storage component")
	ErrReadOnly        = errors.New("Failed write to readonly storage")
)

Graph storage related error types

var (
	ErrInvalidData = errors.New("Invalid data")
	ErrIndexError  = errors.New("Index error")
	ErrReading     = errors.New("Could not read graph information")
	ErrWriting     = errors.New("Could not write graph information")
	ErrRule        = errors.New("Graph rule error")
)

Graph related error types

var CaseSensitiveWordIndex = false

CaseSensitiveWordIndex is a flag to indicate if the index should be case sensitive.

Types

type GraphError

type GraphError struct {
	Type   error  // Error type (to be used for equal checks)
	Detail string // Details of this error
}

GraphError is a graph related error

func (*GraphError) Error

func (ge *GraphError) Error() string

Error returns a human-readable string representation of this error.

type IndexManager

type IndexManager struct {
	// contains filtered or unexported fields
}

IndexManager data structure

func NewIndexManager

func NewIndexManager(htree *hash.HTree) *IndexManager

NewIndexManager creates a new index manager instance.

func (*IndexManager) Count

func (im *IndexManager) Count(attr, word string) (int, error)

Count returns the number of found nodes for a given word in a given attribute.

func (*IndexManager) Deindex

func (im *IndexManager) Deindex(key string, obj map[string]string) error

Deindex deindexes (removes) a given object.

func (*IndexManager) Index

func (im *IndexManager) Index(key string, obj map[string]string) error

Index indexes (inserts) a given object.

func (*IndexManager) LookupPhrase

func (im *IndexManager) LookupPhrase(attr, phrase string) ([]string, error)

LookupPhrase finds all nodes where an attribute contains a certain phrase. This call returns a list of node keys which contain the phrase at least once.

func (*IndexManager) LookupValue

func (im *IndexManager) LookupValue(attr, value string) ([]string, error)

LookupValue finds all nodes where an attribute has a certain value. This call returns a list of node keys.

func (*IndexManager) LookupWord

func (im *IndexManager) LookupWord(attr, word string) (map[string][]uint64, error)

LookupWord finds all nodes where an attribute contains a certain word. This call returns a map which maps node key to a list of word positions.

func (*IndexManager) Reindex

func (im *IndexManager) Reindex(key string, newObj map[string]string,
	oldObj map[string]string) error

Reindex reindexes (updates) a given object.

func (*IndexManager) String

func (im *IndexManager) String() string

String returns a string representation of this index manager.

type NamesManager

type NamesManager struct {
	// contains filtered or unexported fields
}

NamesManager data structure

func NewNamesManager

func NewNamesManager(nameDB map[string]string) *NamesManager

NewNamesManager creates a new names manager instance.

func (*NamesManager) Decode16

func (gs *NamesManager) Decode16(val string) string

Decode16 decodes a given 16 bit string to a value.

func (*NamesManager) Decode32

func (gs *NamesManager) Decode32(val string) string

Decode32 decodes a given 32 bit string to a value.

func (*NamesManager) Encode16

func (gs *NamesManager) Encode16(val string, create bool) string

Encode16 encodes a given value as a 16 bit string. If the create flag is set to false then a new entry will not be created if it does not exist.

func (*NamesManager) Encode32

func (gs *NamesManager) Encode32(val string, create bool) string

Encode32 encodes a given value as a 32 bit string. If the create flag is set to false then a new entry will not be created if it does not exist.