oplog

package
v0.9.11 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 10, 2020 License: GPL-3.0 Imports: 14 Imported by: 0

Documentation

Overview

Package oplog is an operation-based replicated data type of append-only logs oplog has three main structures: logbook, log, and op A log is a sequence of operations attributed to a single author, designated by a private key. an operation is a record of some action an author took. Applications iterate the sequence of operations to produce the current state. Logs can be arranged into hierarchies to form logical groupings. A book contains an author's logs, both logs they've written as well as logs replicated from other authors. Books are encrypted at rest using the author private key.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNotFound is a sentinel error for data not found in a logbook
	ErrNotFound = fmt.Errorf("log: not found")
)

Functions

This section is empty.

Types

type AuthorLogstore added in v0.9.3 

type AuthorLogstore interface {
	// All AuthorLogstores are Logstores
	Logstore

	// get the id of the oplog that represnts this Logstore's author
	ID() string
	// attribute ownership of the logstore to an author
	// the given id MUST equal the id of a log already in the logstore
	SetID(ctx context.Context, id string) error

	// marshals all logs to a slice of bytes encrypted with the given private key
	FlatbufferCipher(pk crypto.PrivKey) ([]byte, error)
	// decrypt flatbuffer bytes, re-hydrating the store
	UnmarshalFlatbufferCipher(ctx context.Context, pk crypto.PrivKey, ciphertext []byte) error
}

AuthorLogstore describes a store owned by a single author, it adds encryption methods for safe local persistence as well as owner ID accessors

type Journal added in v0.9.3 

type Journal struct {
	// contains filtered or unexported fields
}

Journal is a store of logs known to a single author, representing their view of an abstract dataset graph. journals live in memory by default, and can be encrypted for storage

func (*Journal) Children added in v0.9.3 

func (j *Journal) Children(ctx context.Context, l *Log) error

Children gets all descentants of a log, because logbook stores all descendants in memory, children is a proxy for descenants

func (*Journal) Descendants added in v0.9.3 

func (j *Journal) Descendants(ctx context.Context, l *Log) error

Descendants gets all descentants of a log & assigns the results to the given Log parameter, setting only the Logs field

func (Journal) FlatbufferCipher added in v0.9.3 

func (j Journal) FlatbufferCipher(pk crypto.PrivKey) ([]byte, error)

FlatbufferCipher marshals journal to a flatbuffer and encrypts the book using a given private key. This same private key must be retained elsewhere to read the flatbuffer later on

func (*Journal) Get added in v0.9.9 

func (j *Journal) Get(_ context.Context, id string) (*Log, error)

Get fetches a log for a given ID

func (*Journal) HeadRef added in v0.9.3 

func (j *Journal) HeadRef(_ context.Context, names ...string) (*Log, error)

HeadRef traverses the log graph & pulls out a log based on named head references HeadRef will not return logs that have been marked as removed. To fetch removed logs either traverse the entire journal or reference a log by ID

func (*Journal) ID added in v0.9.3 

func (j *Journal) ID() string

ID gets the journal author identifier

func (*Journal) Logs added in v0.9.3 

func (j *Journal) Logs(ctx context.Context, offset, limit int) (topLevel []*Log, err error)

Logs returns the full map of logs keyed by model type

func (*Journal) MergeLog added in v0.9.3 

func (j *Journal) MergeLog(ctx context.Context, l *Log) error

MergeLog adds a log to the journal

func (*Journal) RemoveLog added in v0.9.3 

func (j *Journal) RemoveLog(ctx context.Context, names ...string) error

RemoveLog removes a log from the journal TODO (b5) - this currently won't work when trying to remove the root log

func (*Journal) SetID added in v0.9.3 

func (j *Journal) SetID(ctx context.Context, id string) error

SetID assigns the book identifier

func (*Journal) UnmarshalFlatbufferCipher added in v0.9.3 

func (j *Journal) UnmarshalFlatbufferCipher(ctx context.Context, pk crypto.PrivKey, ciphertext []byte) error

UnmarshalFlatbufferCipher decrypts and loads a flatbuffer ciphertext

type Log 

type Log struct {
	ParentID  string // init id of parent Log
	Signature []byte
	Ops       []Op
	Logs      []*Log
	// contains filtered or unexported fields
}

Log is a causally-ordered set of operations performed by a single author. log attribution is verified by an author's signature

func FromFlatbufferBytes 

func FromFlatbufferBytes(data []byte) (*Log, error)

FromFlatbufferBytes initializes a log from flatbuffer data

func GetWithSparseAncestorsAllDescendants added in v0.9.10 

func GetWithSparseAncestorsAllDescendants(ctx context.Context, store Logstore, id string) (*Log, error)

GetWithSparseAncestorsAllDescendants is a fast-path method for getting a log that includes sparse parents & complete descendants. "sparse parents" means the only children given in parent the returned log will match the ID of the request, with parents

func InitLog 

func InitLog(initop Op) *Log

InitLog creates a Log from an initialization operation

func (*Log) AddChild 

func (lg *Log) AddChild(l *Log)

AddChild appends a log as a direct descendant of this log, controlling for duplicates

func (*Log) Append 

func (lg *Log) Append(op Op)

Append adds an operation to the log

func (Log) Author 

func (lg Log) Author() (identifier string)

Author returns the name and identifier this log is attributed to

func (*Log) DeepCopy added in v0.9.5 

func (lg *Log) DeepCopy() *Log

DeepCopy produces a fresh duplicate of this log

func (Log) FlatbufferBytes added in v0.9.5 

func (lg Log) FlatbufferBytes() []byte

FlatbufferBytes marshals a log to flabuffer-formatted bytes

func (Log) Head 

func (lg Log) Head() Op

Head gets the latest operation in the log

func (*Log) HeadRef 

func (lg *Log) HeadRef(names ...string) (*Log, error)

HeadRef returns a descendant log, traversing the log tree by name HeadRef will not return logs that have been marked as removed. To fetch removed logs either traverse the entire book or reference a log by ID

func (Log) ID 

func (lg Log) ID() string

ID returns the hash of the initialization operation if the log is empty, returns the empty string

func (*Log) Log 

func (lg *Log) Log(id string) (*Log, error)

Log fetches a log by ID, checking the current log and all descendants for an exact match

func (Log) MarshalFlatbuffer 

func (lg Log) MarshalFlatbuffer(builder *flatbuffers.Builder) flatbuffers.UOffsetT

MarshalFlatbuffer writes log to a flatbuffer, returning the ending byte offset

func (*Log) Merge 

func (lg *Log) Merge(l *Log)

Merge combines two logs that are assumed to be a shared root, combining children from both branches, matching branches prefer longer Opsets Merging relies on comparison of initialization operations, which must be present to constitute a match

func (Log) Model 

func (lg Log) Model() uint32

Model gives the operation type for a log, based on the first operation written to the log. Logs can contain multiple models of operations, but the first operation written to a log determines the kind of log for catagorization purposes

func (Log) Name 

func (lg Log) Name() string

Name returns the human-readable name for this log, determined by the initialization event

func (*Log) Parent 

func (lg *Log) Parent() *Log

Parent returns this log's parent if one exists

func (Log) Removed 

func (lg Log) Removed() bool

Removed returns true if the log contains a remove operation for the log model

func (*Log) Sign 

func (lg *Log) Sign(pk crypto.PrivKey) (err error)

Sign assigns the log signature by signing the logging checksum with a given private key TODO (b5) - this is assuming the log is authored by this private key. as soon as we add collaborators, this won't be true

func (Log) SigningBytes 

func (lg Log) SigningBytes() []byte

SigningBytes perpares a byte slice for signing from a log's operations

func (*Log) UnmarshalFlatbuffer 

func (lg *Log) UnmarshalFlatbuffer(lfb *logfb.Log, parent *Log) (err error)

UnmarshalFlatbuffer populates a logfb.Log from a Log pointer

func (*Log) UnmarshalFlatbufferBytes 

func (lg *Log) UnmarshalFlatbufferBytes(data []byte) error

UnmarshalFlatbufferBytes is a convenince wrapper to deserialze a flatbuffer slice into a log

func (Log) Verify 

func (lg Log) Verify(pub crypto.PubKey) error

Verify confirms that the signature for a log matches

type Logstore added in v0.9.3 

type Logstore interface {
	// MergeLog adds a Log to the store, controlling for conflicts
	// * logs that are already known to the store are merged with a
	//   longest-log-wins strategy, adding all descendants
	// * new top level logs are appended to the store, including all descendants
	// * attempting to add a log with a parent not already in the store MUST fail
	//
	// TODO (b5) - currently a Log pointer doesn't provide a clear method for
	// getting the ID of it's parent, which negates the potential for attempting
	// to merge child log, so we don't need to control for the third point quite
	// yet
	MergeLog(ctx context.Context, l *Log) error

	// Remove a log from the store, all descendant logs must be removed as well
	RemoveLog(ctx context.Context, names ...string) error

	// Logs lists top level logs in the store, that is, the set logs that have no
	// parent. passing -1 as a limit returns all top level logs after the offset
	//
	// The order of logs returned is up to the store, but the stored order must
	// be deterministic
	Logs(ctx context.Context, offset, limit int) (topLevel []*Log, err error)

	// get a log according to a hierarchy of log.Name() references
	// for example, fetching HeadRef(ctx, "foo", "bar", "baz") is a request
	// for the log at the hierarchy foo/bar/baz:
	//   foo
	//     bar
	//       baz
	//
	// HeadRef must return ErrNotFound if any name in the heirarchy is  missing
	// from the store
	// Head references are mutated by adding operations to a log that modifies
	// the name of the initialization model, which means names are not a
	// persistent identifier
	//
	// HeadRef MAY return children of a log. If the returned log.Log value is
	// populated, it MUST contain all children of the log.
	// use Logstore.Children or Logstore.Descendants to populate missing children
	HeadRef(ctx context.Context, names ...string) (*Log, error)

	// get a log according to it's ID string
	// Log must return ErrNotFound if the ID does not exist
	//
	// Log MAY return children of a log. If the returned log.Log value is
	// populated, it MUST contain all children of the log.
	// use Logstore.Children or Logstore.Descendants to populate missing children
	Get(ctx context.Context, id string) (*Log, error)

	// get the immediate descendants of a log, using the given log as an outparam.
	// Children must only mutate Logs field of the passed-in log pointer
	// added Children MAY include decendant logs
	Children(ctx context.Context, l *Log) error
	// get all generations of a log, using the given log as an outparam
	// Descendants MUST only mutate Logs field of the passed-in log pointer
	Descendants(ctx context.Context, l *Log) error
}

Logstore persists a set of operations organized into hierarchical append-only logs

type Op 

type Op struct {
	Type      OpType   // type of operation
	Model     uint32   // data model to operate on
	Ref       string   // identifier of data this operation is documenting
	Prev      string   // previous reference in a causal history
	Relations []string // references this operation relates to. usage is operation type-dependant
	Name      string   // human-readable name for the reference
	AuthorID  string   // identifier for author

	Timestamp int64  // operation timestamp, for annotation purposes only
	Size      int64  // size of the referenced value in bytes
	Note      string // operation annotation for users. eg: commit title
}

Op is an operation, a single atomic unit in a log that describes a state change

func UnmarshalOpFlatbuffer 

func UnmarshalOpFlatbuffer(o *logfb.Operation) Op

UnmarshalOpFlatbuffer creates an op from a flatbuffer operation pointer

func (Op) Equal 

func (o Op) Equal(b Op) bool

Equal tests equality between two operations

func (Op) Hash 

func (o Op) Hash() string

Hash returns the base32-lowercase-encoded blake2b-256 hash of the Op flatbuffer

func (Op) MarshalFlatbuffer 

func (o Op) MarshalFlatbuffer(builder *flatbuffers.Builder) flatbuffers.UOffsetT

MarshalFlatbuffer writes this operation to a flatbuffer, returning the ending byte offset

type OpType 

type OpType byte

OpType is the set of all kinds of operations, they are two bytes in length OpType splits the provided byte in half, using the higher 4 bits for the "category" of operation, and the lower 4 bits for the type of operation within the category the second byte is reserved for future use 

const (
	// OpTypeInit is the creation of a model
	OpTypeInit OpType = 0x01
	// OpTypeAmend represents amending a model
	OpTypeAmend OpType = 0x02
	// OpTypeRemove represents deleting a model
	OpTypeRemove OpType = 0x03
)

type SparseAncestorsAllDescendantsLogstore added in v0.9.10 

type SparseAncestorsAllDescendantsLogstore interface {
	Logstore
	// GetSparseAncestorsAllDescendants is a fast-path method for getting a
	// log that includes sparse parents & complete descendants. "sparse parents"
	// have the only children given in parent specified
	// AllDescendants include
	// the returned log will match the ID of the request, with parents
	GetSparseAncestorsAllDescendants(ctx context.Context, id string) (*Log, error)
}

SparseAncestorsAllDescendantsLogstore is a an extension interface to Logstore with an optimized method for getting a log with sparse parents and all descendants

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.