grove

Documentation

Overview

Package grove implements an on-disk storage format for arbor forest nodes. This hierarchical storage format is called a "grove", and the management type implemented by this package satisfies the forest.Store interface.

Note: this package is not yet complete.

Index

• type ChildCache
  • func NewChildCache() *ChildCache
  • func (c *ChildCache) Add(parent *fields.QualifiedHash, children ...*fields.QualifiedHash)
  • func (c *ChildCache) Get(parent *fields.QualifiedHash) ([]*fields.QualifiedHash, bool)
  • func (c *ChildCache) RemoveChild(parent, child *fields.QualifiedHash) error
  • func (c *ChildCache) RemoveParent(id *fields.QualifiedHash)
• type ErrorGroup
  • func (e ErrorGroup) Error() string
• type FS
• type File
• type Grove
  • func New(root string) (*Grove, error)
  • func NewWithFS(fs FS) (*Grove, error)
  • func (g *Grove) Add(node forest.Node) (err error)
  • func (g *Grove) CacheChildInfo(node forest.Node)
  • func (g *Grove) Children(id *fields.QualifiedHash) ([]*fields.QualifiedHash, error)
  • func (g *Grove) CopyInto(other forest.Store) error
  • func (g *Grove) Get(nodeID *fields.QualifiedHash) (node forest.Node, present bool, err error)
  • func (g *Grove) GetCommunity(id *fields.QualifiedHash) (forest.Node, bool, error)
  • func (g *Grove) GetConversation(communityID, conversationID *fields.QualifiedHash) (forest.Node, bool, error)
  • func (g *Grove) GetIdentity(id *fields.QualifiedHash) (forest.Node, bool, error)
  • func (g *Grove) GetReply(communityID, conversationID, replyID *fields.QualifiedHash) (forest.Node, bool, error)
  • func (g *Grove) RebuildChildCache() error
  • func (g *Grove) Recent(nodeType fields.NodeType, quantity int) ([]forest.Node, error)
  • func (g *Grove) RemoveSubtree(id *fields.QualifiedHash) error
  • func (g *Grove) SetCorruptNodeHandler(handler func(string))
• type RelativeFS
  • func (r RelativeFS) Create(path string) (File, error)
  • func (r RelativeFS) Open(path string) (File, error)
  • func (r RelativeFS) OpenFile(path string, flag int, perm os.FileMode) (File, error)
  • func (r RelativeFS) Remove(path string) error
• type UnmarshalError
  • func (e UnmarshalError) Error() string
  • func (e UnmarshalError) Unwrap() error
• Bugs

Types

type ChildCache

type ChildCache struct {
	Elements map[string]map[string]*fields.QualifiedHash
}

ChildCache provides a simple API for keeping track of which node IDs are known to be children of which other node IDs.

func NewChildCache

func NewChildCache() *ChildCache

NewChildCache creates a new empty child cache

func (*ChildCache) Add

func (c *ChildCache) Add(parent *fields.QualifiedHash, children ...*fields.QualifiedHash)

Add inserts the given children as child elements of the given parent.

func (*ChildCache) Get

func (c *ChildCache) Get(parent *fields.QualifiedHash) ([]*fields.QualifiedHash, bool)

Get returns all known children of the given parent. The second return value indicates whether or not the parent was found in the cache.

func (*ChildCache) RemoveChild

func (c *ChildCache) RemoveChild(parent, child *fields.QualifiedHash) error

RemoveChild removes the provided child node from the list of children for the provided parent node.

func (*ChildCache) RemoveParent

func (c *ChildCache) RemoveParent(id *fields.QualifiedHash)

RemoveParent destroys the top-level cache entry for the given node.

type ErrorGroup

type ErrorGroup []error

ErrorGroup wraps multiple errors into a single return value.

func (ErrorGroup) Error

func (e ErrorGroup) Error() string

type FS

type FS interface {
	Open(path string) (File, error)
	Create(path string) (File, error)
	OpenFile(path string, flag int, perm os.FileMode) (File, error)
	Remove(path string) error
}

FS represents a type that acts as a filesystem. It can create and open files at specific paths

type File

type File interface {
	io.ReadWriteCloser
	Name() string
	Readdir(n int) ([]os.FileInfo, error)
}

File represents a type that supports file-like operations. *os.File implements this interface, and will likely be used most of the time. This interface exists mostly to simply testing.

type Grove

type Grove struct {
	FS
	NodeCache *store.MemoryStore
	*ChildCache
	// contains filtered or unexported fields
}

Grove is an on-disk store for arbor forest nodes. It maintains internal in-memory caches in order to accelerate certain expensive operations. Because of this, it must be notified when new content appears on disk. The recommended way to handle this is to use file-system watching on the grove directory and to call Add() with any new nodes that appear (it is not an error to call Add() on a node already present in a store). Another (potentially more expensive) way to ensure consistency in the event of a disk modification is to call RebuildChildCache().

func New

func New(root string) (*Grove, error)

New constructs a Grove that stores nodes in a hierarchy rooted at the given path.

func NewWithFS

func NewWithFS(fs FS) (*Grove, error)

NewWithFS constructs a Grove using the given FS implementation to access its nodes. This is primarily useful for testing.

func (*Grove) Add

func (g *Grove) Add(node forest.Node) (err error)

Add inserts the node into the grove. If the given node is already in the grove, Add will do nothing. It is not an error to insert a node more than once.

func (*Grove) CacheChildInfo

func (g *Grove) CacheChildInfo(node forest.Node)

CacheChildInfo updates the child cache information for the given node.

func (*Grove) Children

func (g *Grove) Children(id *fields.QualifiedHash) ([]*fields.QualifiedHash, error)

Children returns the IDs of all known child nodes of the specified ID. Any error opening, reading, or parsing files in the grove that occurs during the search for child nodes will cause the entire operation to error.

func (*Grove) CopyInto

func (g *Grove) CopyInto(other forest.Store) error

CopyInto copies all nodes from the store into the provided store.

BUG(whereswaldon): this method is not yet implemented. It requires more extensive file manipulation than other Grove methods (listing directory contents) and has therefore been deprioritized in favor of the functionality that can be implemented simply. However, it is implementable, and should be done as soon as is feasible.

func (*Grove) Get

func (g *Grove) Get(nodeID *fields.QualifiedHash) (node forest.Node, present bool, err error)

Get searches the grove for a node with the given id. It returns the node if it was found, a boolean indicating whether it was found, and an error (if there was a problem searching for the node). The returned `present` will never be true unless the returned `node` holds an actual node struct. If the file holding a node exists on disk but was unable to be opened, read, or parsed, `present` will still be false.

func (*Grove) GetCommunity

func (g *Grove) GetCommunity(id *fields.QualifiedHash) (forest.Node, bool, error)

GetCommunity returns an Community node with the given ID (if it is present in the grove). This operation may be faster than using Get, as the grove may be able to do less search work when it knows the type of node you're looking for in advance.

BUG(whereswaldon): The current implementation may return nodes of the wrong NodeType if they match the provided ID

func (*Grove) GetConversation

func (g *Grove) GetConversation(communityID, conversationID *fields.QualifiedHash) (forest.Node, bool, error)

GetConversation returns an Conversation node with the given ID (if it is present in the grove). This operation may be faster than using Get, as the grove may be able to do less search work when it knows the type of node you're looking for and its parent node in advance.

BUG(whereswaldon): The current implementation may return nodes of the wrong NodeType if they match the provided ID

func (*Grove) GetIdentity

func (g *Grove) GetIdentity(id *fields.QualifiedHash) (forest.Node, bool, error)

GetIdentity returns an Identity node with the given ID (if it is present in the grove). This operation may be faster than using Get, as the grove may be able to do less search work when it knows the type of node you're looking for in advance.

BUG(whereswaldon): The current implementation may return nodes of the wrong NodeType if they match the provided ID

func (*Grove) GetReply

func (g *Grove) GetReply(communityID, conversationID, replyID *fields.QualifiedHash) (forest.Node, bool, error)

GetReply returns an Reply node with the given ID (if it is present in the grove). This operation may be faster than using Get, as the grove may be able to do less search work when it knows the type of node you're looking for and its parent community and conversation node in advance.

BUG(whereswaldon): The current implementation may return nodes of the wrong NodeType if they match the provided ID

func (*Grove) RebuildChildCache

func (g *Grove) RebuildChildCache() error

RebuildChildCache must be called each time a node is inserted into the underlying storage without actually calling Add() on the grove. Without this, calls to Children() will not always include new results. This method may return an error if not all nodes in the grove could be read from disk.

func (*Grove) Recent

func (g *Grove) Recent(nodeType fields.NodeType, quantity int) ([]forest.Node, error)

Recent returns a slice of the most recently-created nodes of the given type. The slice is sorted so that the most-recently-created nodes are at the beginning.

NOTE: this function may return both a valid slice of nodes and an error in the case that some nodes failed to be unmarshaled from disk, but others were successful. Calling code should always check whether the node list is empty before throwing it away. If there is an error, it will be of type ErrorGroup.

func (*Grove) RemoveSubtree

func (g *Grove) RemoveSubtree(id *fields.QualifiedHash) error

RemoveSubtree removes the subtree rooted at the node with the provided ID from the grove.

func (*Grove) SetCorruptNodeHandler

func (g *Grove) SetCorruptNodeHandler(handler func(string))

SetCorruptNodeHandler establishes a handler function that will be invoked with the string ID of a node that was detected to be corrupt on disk. The provided function should be as simple and fast as possible in order to ensure good performance for the grove.

type RelativeFS

type RelativeFS struct {
	Root string
}

RelativeFS is a file system that acts relative to a specific path

func (RelativeFS) Create

func (r RelativeFS) Create(path string) (File, error)

Create makes the given path as an absolute path relative to the root of the RelativeFS

func (RelativeFS) Open

func (r RelativeFS) Open(path string) (File, error)

Open opens the given path as an absolute path relative to the root of the RelativeFS

func (RelativeFS) OpenFile

func (r RelativeFS) OpenFile(path string, flag int, perm os.FileMode) (File, error)

OpenFile opens the given path as an absolute path relative to the root of the RelativeFS

func (RelativeFS) Remove

func (r RelativeFS) Remove(path string) error

Remove removes the given path relative to the root of the RelativeFS.

type UnmarshalError

type UnmarshalError struct {
	Reason error
	Node   string
}

func (UnmarshalError) Error

func (e UnmarshalError) Error() string

func (UnmarshalError) Unwrap

func (e UnmarshalError) Unwrap() error

Notes

Bugs

• The current implementation may return nodes of the wrong NodeType if they match the provided ID

• The current implementation may return nodes of the wrong NodeType if they match the provided ID

• The current implementation may return nodes of the wrong NodeType if they match the provided ID

• The current implementation may return nodes of the wrong NodeType if they match the provided ID

• this method is not yet implemented. It requires more extensive file manipulation than other Grove methods (listing directory contents) and has therefore been deprioritized in favor of the functionality that can be implemented simply. However, it is implementable, and should be done as soon as is feasible.