datamodel

package
v0.18.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 1, 2022 License: MIT Imports: 4 Imported by: 259

Documentation

Overview

The datamodel package defines the most essential interfaces for describing IPLD Data -- such as Node, NodePrototype, NodeBuilder, Link, and Path.

Note that since interfaces in this package are the core of the library, choices made here maximize correctness and performance -- these choices are *not* always the choices that would maximize ergonomics. (Ergonomics can come on top; performance generally can't.) You'll want to check out other packages for functions with more ergonomics; for example, 'fluent' and its subpackages provide lots of ways to work with data; 'traversal' provides some ergonomic features for walking around data graphs; any use of schemas will provide a bunch of useful data validation options; or you can make your own function decorators that do what *you* need.

Index

Constants

This section is empty.

Variables

View Source 
var (
	KindSet_Recursive = KindSet{Kind_Map, Kind_List}
	KindSet_Scalar    = KindSet{Kind_Null, Kind_Bool, Kind_Int, Kind_Float, Kind_String, Kind_Bytes, Kind_Link}

	KindSet_JustMap    = KindSet{Kind_Map}
	KindSet_JustList   = KindSet{Kind_List}
	KindSet_JustNull   = KindSet{Kind_Null}
	KindSet_JustBool   = KindSet{Kind_Bool}
	KindSet_JustInt    = KindSet{Kind_Int}
	KindSet_JustFloat  = KindSet{Kind_Float}
	KindSet_JustString = KindSet{Kind_String}
	KindSet_JustBytes  = KindSet{Kind_Bytes}
	KindSet_JustLink   = KindSet{Kind_Link}
)

Functions

func Copy added in v0.14.0 

func Copy(n Node, na NodeAssembler) error

Copy does an explicit shallow copy of a Node's data into a NodeAssembler.

This can be used to flip data from one memory layout to another (for example, from basicnode to using using bindnode, or to codegenerated node implementations, or to or from ADL nodes, etc).

The copy is implemented by ranging over the contents if it's a recursive kind, and for each of them, using `AssignNode` on the child values; for scalars, it's just calling the appropriate `Assign*` method.

Many NodeAssembler implementations use this as a fallback behavior in their `AssignNode` method (that is, they call to this function after all other special faster shortcuts they might prefer to employ, such as direct struct copying if they share internal memory layouts, etc, have been tried already).

func DeepEqual 

func DeepEqual(x, y Node) bool

DeepEqual reports whether x and y are "deeply equal" as IPLD nodes. This is similar to reflect.DeepEqual, but based around the Node interface.

Two nodes must have the same kind to be deeply equal. If either node has the invalid kind, the nodes are not deeply equal.

Two nodes of scalar kinds (null, bool, int, float, string, bytes, link) are deeply equal if their Go values, as returned by AsKind methods, are equal as per Go's == comparison operator.

Note that Links are compared in a shallow way, without being followed. This will generally be enough, as it's rare to have two different links to the same IPLD data by using a different codec or multihash type.

Two nodes of recursive kinds (map, list) must have the same length to be deeply equal. Their elements, as reported by iterators, must be deeply equal. The elements are compared in the iterator's order, meaning two maps sorting the same keys differently might not be equal.

Note that this function panics if either Node returns an error. We only call valid methods for each Kind, so an error should only happen if a Node implementation breaks that contract. It is generally not recommended to call DeepEqual on ADL nodes.

Types

type ErrInvalidSegmentForList 

type ErrInvalidSegmentForList struct {
	// TypeName may indicate the named type of a node the function was called on,
	// or be empty string if working on untyped data.
	TypeName string

	// TroubleSegment is the segment we couldn't use.
	TroubleSegment PathSegment

	// Reason may explain more about why the PathSegment couldn't be used;
	// in practice, it's probably a 'strconv.NumError'.
	Reason error
}

ErrInvalidSegmentForList is returned when using Node.LookupBySegment and the given PathSegment can't be applied to a list because it's unparsable as a number.

func (ErrInvalidSegmentForList) Error 

func (e ErrInvalidSegmentForList) Error() string

type ErrIteratorOverread 

type ErrIteratorOverread struct{}

ErrIteratorOverread is returned when calling 'Next' on a MapIterator or ListIterator when it is already done.

func (ErrIteratorOverread) Error 

func (e ErrIteratorOverread) Error() string

type ErrNotExists 

type ErrNotExists struct {
	Segment PathSegment
}

ErrNotExists may be returned from the lookup functions of the Node interface to indicate a missing value.

Note that schema.ErrNoSuchField is another type of error which sometimes occurs in similar places as ErrNotExists. ErrNoSuchField is preferred when handling data with constraints provided by a schema that mean that a field can *never* exist (as differentiated from a map key which is simply absent in some data).

func (ErrNotExists) Error 

func (e ErrNotExists) Error() string

type ErrRepeatedMapKey 

type ErrRepeatedMapKey struct {
	Key Node
}

ErrRepeatedMapKey is an error indicating that a key was inserted into a map that already contains that key.

This error may be returned by any methods that add data to a map -- any of the methods on a NodeAssembler that was yielded by MapAssembler.AssignKey(), or from the MapAssembler.AssignDirectly() method.

func (ErrRepeatedMapKey) Error 

func (e ErrRepeatedMapKey) Error() string

type ErrWrongKind 

type ErrWrongKind struct {
	// TypeName may optionally indicate the named type of a node the function
	// was called on (if the node was typed!), or, may be the empty string.
	TypeName string

	// MethodName is literally the string for the operation attempted, e.g.
	// "AsString".
	//
	// For methods on nodebuilders, we say e.g. "NodeBuilder.CreateMap".
	MethodName string

	// ApprorpriateKind describes which Kinds the erroring method would
	// make sense for.
	AppropriateKind KindSet

	// ActualKind describes the Kind of the node the method was called on.
	//
	// In the case of typed nodes, this will typically refer to the 'natural'
	// data-model kind for such a type (e.g., structs will say 'map' here).
	ActualKind Kind
}

ErrWrongKind may be returned from functions on the Node interface when a method is invoked which doesn't make sense for the Kind that node concretely contains.

For example, calling AsString on a map will return ErrWrongKind. Calling Lookup on an int will similarly return ErrWrongKind.

func (ErrWrongKind) Error 

func (e ErrWrongKind) Error() string

type Kind 

type Kind uint8

Kind represents the primitive kind in the IPLD data model. All of these kinds map directly onto serializable data.

Note that Kind contains the concept of "map", but not "struct" or "object" -- those are a concepts that could be introduced in a type system layers, but are *not* present in the data model layer, and therefore they aren't included in the Kind enum. 

const (
	Kind_Invalid Kind = 0
	Kind_Map     Kind = '{'
	Kind_List    Kind = '['
	Kind_Null    Kind = '0'
	Kind_Bool    Kind = 'b'
	Kind_Int     Kind = 'i'
	Kind_Float   Kind = 'f'
	Kind_String  Kind = 's'
	Kind_Bytes   Kind = 'x'
	Kind_Link    Kind = '/'
)

func (Kind) String 

func (k Kind) String() string

type KindSet 

type KindSet []Kind

KindSet is a type with a few enumerated consts that are commonly used (mostly, in error messages).

func (KindSet) Contains 

func (x KindSet) Contains(e Kind) bool

func (KindSet) String 

func (x KindSet) String() string

type LargeBytesNode added in v0.16.0 

type LargeBytesNode interface {
	Node
	// AsLargeBytes returns an io.ReadSeeker that can be used to read the contents of the node.
	// Note that the presence of this method / interface does not imply that the node
	// can always return a valid io.ReadSeeker, and the error value must also be checked
	// for support.
	// It is not guaranteed that all implementations will implement the full semantics of
	// Seek, in particular, they may refuse to seek to the end of a large bytes node if
	// it is not possible to do so efficiently.
	// The io.ReadSeeker returned by AsLargeBytes must be a seperate instance from subsequent
	// calls to AsLargeBytes. Calls to read or seek on one returned instance should NOT
	// affect the read position of other returned instances.
	AsLargeBytes() (io.ReadSeeker, error)
}

LargeBytesNode is an optional interface extending a Bytes node that allows its contents to be accessed through an io.ReadSeeker instead of a []byte slice. Use of an io.Reader is encouraged, as it allows for streaming large byte slices without allocating a large slice in memory. 

type Link interface {
	// Prototype should return a LinkPrototype which carries the information
	// to make more Link values similar to this one (but with different hashes).
	Prototype() LinkPrototype

	// String should return a reasonably human-readable debug-friendly representation the Link.
	// There is no contract that requires that the string be able to be parsed back into a Link value,
	// but the string should be unique (e.g. not elide any parts of the hash).
	String() string

	// Binary should return the densest possible encoding of the Link.
	// The value need not be printable or human-readable;
	// the golang string type is used for immutability and for ease of use as a map key.
	// As with the String method, the returned value may not elide any parts of the hash.
	//
	// Note that there is still no contract that the returned value be able to be parsed back into a Link value;
	// not even in the case of `lnk.Prototype().BuildLink(lnk.Binary()[:])`.
	// This is because the value returned by this method may contain data that the LinkPrototype would also restate.
	// (For a concrete example: if using CIDs, this method will return a binary string that includes
	// the cid version indicator, the multicodec and multihash indicators, etc, in addition to the hash itself --
	// whereas the LinkPrototype.BuildLink function still expects to receive only the hash itself alone.)
	Binary() string
}

Link is a special kind of value in IPLD which can be "loaded" to access more nodes.

Nodes can be a Link: "link" is one of the kinds in the IPLD Data Model; and accordingly there is an `ipld.Kind_Link` enum value, and Node has an `AsLink` method.

Links are considered a scalar value in the IPLD Data Model, but when "loaded", the result can be any other IPLD kind: maps, lists, strings, etc.

Link is an interface in the go-ipld implementation, but the most common instantiation of it comes from the `linking/cid` package, and represents CIDs (see https://github.com/multiformats/cid).

The Link interface says very little by itself; it's generally necessary to use type assertions to unpack more specific forms of data. The only real contract is that the Link must be able to return a LinkPrototype, which must be able to produce new Link values of a similar form. (In practice: if you're familiar with CIDs: Link.Prototype is analogous to cid.Prefix.)

The traversal package contains powerful features for walking through large graphs of Nodes while automatically loading and traversing links as the walk goes.

Note that the Link interface should typically be inhabited by a struct or string, as opposed to a pointer. This is because Link is often desirable to be able to use as a golang map key, and in that context, pointers would not result in the desired behavior.

type LinkPrototype 

type LinkPrototype interface {
	// BuildLink should return a new Link value based on the given hashsum.
	// The hashsum argument should typically be a value returned from a
	// https://golang.org/pkg/hash/#Hash.Sum call.
	//
	// The hashsum reference must not be retained (the caller is free to reuse it).
	BuildLink(hashsum []byte) Link
}

LinkPrototype encapsulates any implementation details and parameters necessary for creating a Link, expect for the hash result itself.

LinkPrototype, like Link, is an interface in go-ipld, but the most common instantiation of it comes from the `linking/cid` package, and represents CIDs (see https://github.com/multiformats/cid). If using CIDs as an implementation, LinkPrototype will encapsulate information like multihashType, multicodecType, and cidVersion, for example. (LinkPrototype is analogous to cid.Prefix.)

type ListAssembler 

type ListAssembler interface {
	AssembleValue() NodeAssembler

	Finish() error

	// ValuePrototype returns a NodePrototype that knows how to build values this map can contain.
	//
	// You often don't need this (because you should be able to
	// just feed data and check errors), but it's here.
	//
	// ValuePrototype, much like the matching method on the MapAssembler interface,
	// requires a parameter specifying the index in the list in order to say
	// what NodePrototype will be acceptable as a value at that position.
	// For many lists (and *all* lists which operate exclusively at the Data Model level),
	// this will return the same NodePrototype regardless of the value of 'idx';
	// the only time this value will vary is when operating with a Schema,
	// and handling the representation NodeAssembler for a struct type with
	// a representation of a list kind.
	// If you know you are operating in a situation that won't have varying
	// NodePrototypes, it is acceptable to call `ValuePrototype(0)` and use the
	// resulting NodePrototype for all reasoning.
	ValuePrototype(idx int64) NodePrototype
}

type ListIterator 

type ListIterator interface {
	// Next returns the next index and value.
	//
	// An error value can also be returned at any step: in the case of advanced
	// data structures with incremental loading, it's possible to encounter
	// cancellation or I/O errors at any point in iteration.
	// If an error will be returned by the next call to Next,
	// then the boolean returned by the Done method will be false
	// (meaning it's acceptable to check Done first and move on if it's true,
	// since that both means the iterator is complete and that there is no error).
	// If an error is returned, the key and value may be nil.
	Next() (idx int64, value Node, err error)

	// Done returns false as long as there's at least one more entry to iterate.
	// When Done returns false, iteration can stop.
	//
	// Note when implementing iterators for advanced data layouts (e.g. more than
	// one chunk of backing data, which is loaded incrementally): if your
	// implementation does any I/O during the Done method, and it encounters
	// an error, it must return 'false', so that the following Next call
	// has an opportunity to return the error.
	Done() bool
}

ListIterator is an interface for traversing list nodes. Sequential calls to Next() will yield index-value pairs; Done() describes whether iteration should continue.

ListIterator's Next method returns an index for convenience, but this number will always start at 0 and increment by 1 monotonically. A loop which iterates from 0 to Node.Length while calling Node.LookupByIndex is equivalent to using a ListIterator.

type MapAssembler 

type MapAssembler interface {
	AssembleKey() NodeAssembler   // must be followed by call to AssembleValue.
	AssembleValue() NodeAssembler // must be called immediately after AssembleKey.

	AssembleEntry(k string) (NodeAssembler, error) // shortcut combining AssembleKey and AssembleValue into one step; valid when the key is a string kind.

	Finish() error

	// KeyPrototype returns a NodePrototype that knows how to build keys of a type this map uses.
	//
	// You often don't need this (because you should be able to
	// just feed data and check errors), but it's here.
	//
	// For all Data Model maps, this will answer with a basic concept of "string".
	// For Schema typed maps, this may answer with a more complex type
	// (potentially even a struct type or union type -- anything that can have a string representation).
	KeyPrototype() NodePrototype

	// ValuePrototype returns a NodePrototype that knows how to build values this map can contain.
	//
	// You often don't need this (because you should be able to
	// just feed data and check errors), but it's here.
	//
	// ValuePrototype requires a parameter describing the key in order to say what
	// NodePrototype will be acceptable as a value for that key, because when using
	// struct types (or union types) from the Schemas system, they behave as maps
	// but have different acceptable types for each field (or member, for unions).
	// For plain maps (that is, not structs or unions masquerading as maps),
	// the empty string can be used as a parameter, and the returned NodePrototype
	// can be assumed applicable for all values.
	// Using an empty string for a struct or union will return nil,
	// as will using any string which isn't a field or member of those types.
	//
	// (Design note: a string is sufficient for the parameter here rather than
	// a full Node, because the only cases where the value types vary are also
	// cases where the keys may not be complex.)
	ValuePrototype(k string) NodePrototype
}

MapAssembler assembles a map node! (You guessed it.)

Methods on MapAssembler must be called in a valid order: assemble a key, then assemble a value, then loop as long as desired; when finished, call 'Finish'.

Incorrect order invocations will panic. Calling AssembleKey twice in a row will panic; calling AssembleValue before finishing using the NodeAssembler from AssembleKey will panic; calling AssembleValue twice in a row will panic; etc.

Note that the NodeAssembler yielded from AssembleKey has additional behavior: if the node assembled there matches a key already present in the map, that assembler will emit the error!

type MapIterator 

type MapIterator interface {
	// Next returns the next key-value pair.
	//
	// An error value can also be returned at any step: in the case of advanced
	// data structures with incremental loading, it's possible to encounter
	// cancellation or I/O errors at any point in iteration.
	// If an error will be returned by the next call to Next,
	// then the boolean returned by the Done method will be false
	// (meaning it's acceptable to check Done first and move on if it's true,
	// since that both means the iterator is complete and that there is no error).
	// If an error is returned, the key and value may be nil.
	Next() (key Node, value Node, err error)

	// Done returns false as long as there's at least one more entry to iterate.
	// When Done returns true, iteration can stop.
	//
	// Note when implementing iterators for advanced data layouts (e.g. more than
	// one chunk of backing data, which is loaded incrementally): if your
	// implementation does any I/O during the Done method, and it encounters
	// an error, it must return 'false', so that the following Next call
	// has an opportunity to return the error.
	Done() bool
}

MapIterator is an interface for traversing map nodes. Sequential calls to Next() will yield key-value pairs; Done() describes whether iteration should continue.

Iteration order is defined to be stable: two separate MapIterator created to iterate the same Node will yield the same key-value pairs in the same order. The order itself may be defined by the Node implementation: some Nodes may retain insertion order, and some may return iterators which always yield data in sorted order, for example.

type Node 

type Node interface {
	// Kind returns a value from the Kind enum describing what the
	// essential serializable kind of this node is (map, list, integer, etc).
	// Most other handling of a node requires first switching upon the kind.
	Kind() Kind

	// LookupByString looks up a child object in this node and returns it.
	// The returned Node may be any of the Kind:
	// a primitive (string, int64, etc), a map, a list, or a link.
	//
	// If the Kind of this Node is not Kind_Map, a nil node and an error
	// will be returned.
	//
	// If the key does not exist, a nil node and an error will be returned.
	LookupByString(key string) (Node, error)

	// LookupByNode is the equivalent of LookupByString, but takes a reified Node
	// as a parameter instead of a plain string.
	// This mechanism is useful if working with typed maps (if the key types
	// have constraints, and you already have a reified `schema.TypedNode` value,
	// using that value can save parsing and validation costs);
	// and may simply be convenient if you already have a Node value in hand.
	//
	// (When writing generic functions over Node, a good rule of thumb is:
	// when handling a map, check for `schema.TypedNode`, and in this case prefer
	// the LookupByNode(Node) method; otherwise, favor LookupByString; typically
	// implementations will have their fastest paths thusly.)
	LookupByNode(key Node) (Node, error)

	// LookupByIndex is the equivalent of LookupByString but for indexing into a list.
	// As with LookupByString, the returned Node may be any of the Kind:
	// a primitive (string, int64, etc), a map, a list, or a link.
	//
	// If the Kind of this Node is not Kind_List, a nil node and an error
	// will be returned.
	//
	// If idx is out of range, a nil node and an error will be returned.
	LookupByIndex(idx int64) (Node, error)

	// LookupBySegment is will act as either LookupByString or LookupByIndex,
	// whichever is contextually appropriate.
	//
	// Using LookupBySegment may imply an "atoi" conversion if used on a list node,
	// or an "itoa" conversion if used on a map node.  If an "itoa" conversion
	// takes place, it may error, and this method may return that error.
	LookupBySegment(seg PathSegment) (Node, error)

	// MapIterator returns an iterator which yields key-value pairs
	// traversing the node.
	// If the node kind is anything other than a map, nil will be returned.
	//
	// The iterator will yield every entry in the map; that is, it
	// can be expected that itr.Next will be called node.Length times
	// before itr.Done becomes true.
	MapIterator() MapIterator

	// ListIterator returns an iterator which traverses the node and yields indicies and list entries.
	// If the node kind is anything other than a list, nil will be returned.
	//
	// The iterator will yield every entry in the list; that is, it
	// can be expected that itr.Next will be called node.Length times
	// before itr.Done becomes true.
	//
	// List iteration is ordered, and indices yielded during iteration will range from 0 to Node.Length-1.
	// (The IPLD Data Model definition of lists only defines that it is an ordered list of elements;
	// the definition does not include a concept of sparseness, so the indices are always sequential.)
	ListIterator() ListIterator

	// Length returns the length of a list, or the number of entries in a map,
	// or -1 if the node is not of list nor map kind.
	Length() int64

	// Absent nodes are returned when traversing a struct field that is
	// defined by a schema but unset in the data.  (Absent nodes are not
	// possible otherwise; you'll only see them from `schema.TypedNode`.)
	// The absent flag is necessary so iterating over structs can
	// unambiguously make the distinction between values that are
	// present-and-null versus values that are absent.
	//
	// Absent nodes respond to `Kind()` as `ipld.Kind_Null`,
	// for lack of any better descriptive value; you should therefore
	// always check IsAbsent rather than just a switch on kind
	// when it may be important to handle absent values distinctly.
	IsAbsent() bool

	IsNull() bool
	AsBool() (bool, error)
	AsInt() (int64, error)
	AsFloat() (float64, error)
	AsString() (string, error)
	AsBytes() ([]byte, error)
	AsLink() (Link, error)

	// Prototype returns a NodePrototype which can describe some properties of this node's implementation,
	// and also be used to get a NodeBuilder,
	// which can be use to create new nodes with the same implementation as this one.
	//
	// For typed nodes, the NodePrototype will also implement schema.Type.
	//
	// For Advanced Data Layouts, the NodePrototype will encapsulate any additional
	// parameters and configuration of the ADL, and will also (usually)
	// implement NodePrototypeSupportingAmend.
	//
	// Calling this method should not cause an allocation.
	Prototype() NodePrototype
}

Node represents a value in IPLD. Any point in a tree of data is a node: scalar values (like int64, string, etc) are nodes, and so are recursive values (like map and list).

Nodes and kinds are described in the IPLD specs at https://github.com/ipld/specs/blob/master/data-model-layer/data-model.md .

Methods on the Node interface cover the superset of all possible methods for all possible kinds -- but some methods only make sense for particular kinds, and thus will only make sense to call on values of the appropriate kind. (For example, 'Length' on an integer doesn't make sense, and 'AsInt' on a map certainly doesn't work either!) Use the Kind method to find out the kind of value before calling kind-specific methods. Individual method documentation state which kinds the method is valid for. (If you're familiar with the stdlib reflect package, you'll find the design of the Node interface very comparable to 'reflect.Value'.)

The Node interface is read-only. All of the methods on the interface are for examining values, and implementations should be immutable. The companion interface, NodeBuilder, provides the matching writable methods, and should be use to create a (thence immutable) Node.

Keeping Node immutable and separating mutation into NodeBuilder makes it possible to perform caching (or rather, memoization, since there's no such thing as cache invalidation for immutable systems) of computed properties of Node; use copy-on-write algorithms for memory efficiency; and to generally build pleasant APIs. Many library functions will rely on the immutability of Node (e.g., assuming that pointer-equal nodes do not change in value over time), so any user-defined Node implementations should be careful to uphold the immutability contract.)

There are many different concrete types which implement Node. The primary purpose of various node implementations is to organize memory in the program in different ways -- some in-memory layouts may be more optimal for some programs than others, and changing the Node (and NodeBuilder) implementations lets the programmer choose.

For concrete implementations of Node, check out the "./node/" folder, and the packages within it. "node/basicnode" should probably be your first start; the Node and NodeBuilder implementations in that package work for any data. Other packages are optimized for specific use-cases. Codegen tools can also be used to produce concrete implementations of Node; these may be specific to certain data, but still conform to the Node interface for interoperability and to support higher-level functions.

Nodes may also be *typed* -- see the 'schema' package and `schema.TypedNode` interface, which extends the Node interface with additional methods. Typed nodes have additional constraints and behaviors: for example, they may be a "struct" and have a specific type/structure to what data you can put inside them, but still behave as a regular Node in all ways this interface specifies (so you can traverse typed nodes, etc, without any additional special effort). 

var Absent Node = absentNode{}

Absent is the _other_ kind of node (besides Null) we can have a true singleton implementation of. This is the singleton value for Absent. The Absent Node has Kind() == Kind_Null, returns IsNull() == false (!), returns IsAbsent() == true, returns ErrWrongKind to most other inquiries (as you'd expect), and returns a NodePrototype with a NewBuilder method that simply panics (because why would you ever try to build a new "nothing"?).

Despite its presence in the datamodel package, "absent" is not really a data model concept. Absent should not really be seen in any datamodel Node implementations, for example. Absent is seen used in the realm of schemas and typed data, because there, there's a concept of data that has been described, and yet is not currently present; it is this concept that "absent" is used to express. Absent also sometimes shows up as a distinct case in codecs or other diagnostic printing, and suchlike miscellaneous places; it is for these reasons that it's here in the datamodel package, because it would end up imported somewhat universally for those diagnostic purposes anyway. (This may be worth a design review at some point, but holds up well enough for now.) 

var Null Node = nullNode{}

Null is the one kind of node we can have a true singleton implementation of. This is that value. The Null Node has Kind() == Kind_Null, returns IsNull() == true, returns ErrWrongKind to most other inquiries (as you'd expect), and returns a NodePrototype with a NewBuilder method that simply panics (because why would you ever try to build a new "null"?).

type NodeAssembler 

type NodeAssembler interface {
	BeginMap(sizeHint int64) (MapAssembler, error)
	BeginList(sizeHint int64) (ListAssembler, error)
	AssignNull() error
	AssignBool(bool) error
	AssignInt(int64) error
	AssignFloat(float64) error
	AssignString(string) error
	AssignBytes([]byte) error
	AssignLink(Link) error

	AssignNode(Node) error // if you already have a completely constructed subtree, this method puts the whole thing in place at once.

	// Prototype returns a NodePrototype describing what kind of value we're assembling.
	//
	// You often don't need this (because you should be able to
	// just feed data and check errors), but it's here.
	//
	// Using `this.Prototype().NewBuilder()` to produce a new `Node`,
	// then giving that node to `this.AssignNode(n)` should always work.
	// (Note that this is not necessarily an _exclusive_ statement on what
	// sort of values will be accepted by `this.AssignNode(n)`.)
	Prototype() NodePrototype
}

NodeAssembler is the interface that describes all the ways we can set values in a node that's under construction.

A NodeAssembler is about filling in data. To create a new Node, you should start with a NodeBuilder (which contains a superset of the NodeAssembler methods, and can return the finished Node from its `Build` method). While continuing to build a recursive structure from there, you'll see NodeAssembler for all the child values.

For filling scalar data, there's a `Assign{Kind}` method for each kind; after calling one of these methods, the data is filled in, and the assembler is done. For recursives, there are `BeginMap` and `BeginList` methods, which return an object that needs further manipulation to fill in the contents.

There is also one special method: `AssignNode`. `AssignNode` takes another `Node` as a parameter, and should should internally call one of the other `Assign*` or `Begin*` (and subsequent) functions as appropriate for the kind of the `Node` it is given. This is roughly equivalent to using the `Copy` function (and is often implemented using it!), but `AssignNode` may also try to take faster shortcuts in some implementations, when it detects they're possible. (For example, for typed nodes, if they're the same type, lots of checking can be skipped. For nodes implemented with pointers, lots of copying can be skipped. For nodes that can detect the argument has the same memory layout, faster copy mechanisms can be used; etc.)

Why do both this and the NodeBuilder interface exist? In short: NodeBuilder is when you want to cause an allocation; NodeAssembler can be used to just "fill in" memory. (In the internal gritty details: separate interfaces, one of which lacks a `Build` method, helps us write efficient library internals: avoiding the requirement to be able to return a Node at any random point in the process relieves internals from needing to implement 'freeze' features. This is useful in turn because implementing those 'freeze' features in a language without first-class/compile-time support for them (as golang is) would tend to push complexity and costs to execution time; we'd rather not.)

type NodeBuilder 

type NodeBuilder interface {
	NodeAssembler

	// Build returns the new value after all other assembly has been completed.
	//
	// A method on the NodeAssembler that finishes assembly of the data must
	// be called first (e.g., any of the "Assign*" methods, or "Finish" if
	// the assembly was for a map or a list); that finishing method still has
	// all responsibility for validating the assembled data and returning
	// any errors from that process.
	// (Correspondingly, there is no error return from this method.)
	//
	// Note that building via a representation-level NodePrototype or NodeBuilder
	// returns a node at the type level which implements schema.TypedNode.
	// To obtain the representation-level node, you can do:
	//
	//    // builder is at the representation level, so it returns typed nodes
	//    node := builder.Build().(schema.TypedNode)
	//    reprNode := node.Representation()
	Build() Node

	// Resets the builder.  It can hereafter be used again.
	// Reusing a NodeBuilder can reduce allocations and improve performance.
	//
	// Only call this if you're going to reuse the builder.
	// (Otherwise, it's unnecessary, and may cause an unwanted allocation).
	Reset()
}

type NodePrototype 

type NodePrototype interface {
	// NewBuilder returns a NodeBuilder that can be used to create a new Node.
	//
	// Note that calling NewBuilder often performs an allocation
	// (while in contrast, getting a NodePrototype typically does not!) --
	// this may be consequential when writing high performance code.
	NewBuilder() NodeBuilder
}

NodePrototype describes a node implementation (all Node have a NodePrototype), and a NodePrototype can always be used to get a NodeBuilder.

A NodePrototype may also provide other information about implementation; such information is specific to this library ("prototype" isn't a concept you'll find in the IPLD Specifications), and is usually provided through feature-detection interfaces (for example, see NodePrototypeSupportingAmend).

Generic algorithms for working with IPLD Nodes make use of NodePrototype to get builders for new nodes when creating data, and can also use the feature-detection interfaces to help decide what kind of operations will be optimal to use on a given node implementation.

Note that NodePrototype is not the same as schema.Type. NodePrototype is a (golang-specific!) way to reflect upon the implementation and in-memory layout of some IPLD data. schema.Type is information about how a group of nodes is related in a schema (if they have one!) and the rules that the type mandates the node must follow. (Every node must have a prototype; but schema types are an optional feature.)

type NodePrototypeSupportingAmend 

type NodePrototypeSupportingAmend interface {
	AmendingBuilder(base Node) NodeBuilder
}

NodePrototypeSupportingAmend is a feature-detection interface that can be used on a NodePrototype to see if it's possible to build new nodes of this style while sharing some internal data in a copy-on-write way.

For example, Nodes using an Advanced Data Layout will typically support this behavior, and since ADLs are often used for handling large volumes of data, detecting and using this feature can result in significant performance savings.

type Path 

type Path struct {
	// contains filtered or unexported fields
}

Path describes a series of steps across a tree or DAG of Node, where each segment in the path is a map key or list index (literaly, Path is a slice of PathSegment values). Path is used in describing progress in a traversal; and can also be used as an instruction for traversing from one Node to another. Path values will also often be encountered as part of error messages.

(Note that Paths are useful as an instruction for traversing from *one* Node to *one* other Node; to do a walk from one Node and visit *several* Nodes based on some sort of pattern, look to IPLD Selectors, and the 'traversal/selector' package in this project.)

Path values are always relative. Observe how 'traversal.Focus' requires both a Node and a Path argument -- where to start, and where to go, respectively. Similarly, error values which include a Path will be speaking in reference to the "starting Node" in whatever context they arose from.

The canonical form of a Path is as a list of PathSegment. Each PathSegment is a string; by convention, the string should be in UTF-8 encoding and use NFC normalization, but all operations will regard the string as its constituent eight-bit bytes.

There are no illegal or magical characters in IPLD Paths (in particular, do not mistake them for UNIX system paths). IPLD Paths can only go down: that is, each segment must traverse one node. There is no ".." which means "go up"; and there is no "." which means "stay here". IPLD Paths have no magic behavior around characters such as "~". IPLD Paths do not have a concept of "globs" nor behave specially for a path segment string of "*" (but you may wish to see 'Selectors' for globbing-like features that traverse over IPLD data).

An empty string is a valid PathSegment. (This leads to some unfortunate complications when wishing to represent paths in a simple string format; however, consider that maps do exist in serialized data in the wild where an empty string is used as the key: it is important we be able to correctly describe and address this!)

A string containing "/" (or even being simply "/"!) is a valid PathSegment. (As with empty strings, this is unfortunate (in particular, because it very much doesn't match up well with expectations popularized by UNIX-like filesystems); but, as with empty strings, maps which contain such a key certainly exist, and it is important that we be able to regard them!)

A string starting, ending, or otherwise containing the NUL (\x00) byte is also a valid PathSegment. This follows from the rule of "a string is regarded as its constituent eight-bit bytes": an all-zero byte is not exceptional. In golang, this doesn't pose particular difficulty, but note this would be of marked concern for languages which have "C-style nul-terminated strings".

For an IPLD Path to be represented as a string, an encoding system including escaping is necessary. At present, there is not a single canonical specification for such an escaping; we expect to decide one in the future, but this is not yet settled and done. (This implementation has a 'String' method, but it contains caveats and may be ambiguous for some content. This may be fixed in the future.)

func NewPath 

func NewPath(segments []PathSegment) Path

NewPath returns a Path composed of the given segments.

This constructor function does a defensive copy, in case your segments slice should mutate in the future. (Use NewPathNocopy if this is a performance concern, and you're sure you know what you're doing.)

func NewPathNocopy 

func NewPathNocopy(segments []PathSegment) Path

NewPathNocopy is identical to NewPath but trusts that the segments slice you provide will not be mutated.

func ParsePath 

func ParsePath(pth string) Path

ParsePath converts a string to an IPLD Path, doing a basic parsing of the string using "/" as a delimiter to produce a segmented Path. This is a handy, but not a general-purpose nor spec-compliant (!), way to create a Path: it cannot represent all valid paths.

Multiple subsequent "/" characters will be silently collapsed. E.g., `"foo///bar"` will be treated equivalently to `"foo/bar"`. Prefixed and suffixed extraneous "/" characters are also discarded. This makes this constructor incapable of handling some possible Path values (specifically: paths with empty segements cannot be created with this constructor).

There is no escaping mechanism used by this function. This makes this constructor incapable of handling some possible Path values (specifically, a path segment containing "/" cannot be created, because it will always be intepreted as a segment separator).

No other "cleaning" of the path occurs. See the documentation of the Path struct; in particular, note that ".." does not mean "go up", nor does "." mean "stay here" -- correspondingly, there isn't anything to "clean" in the same sense as 'filepath.Clean' from the standard library filesystem path packages would.

If the provided string contains unprintable characters, or non-UTF-8 or non-NFC-canonicalized bytes, no remark will be made about this, and those bytes will remain part of the PathSegments in the resulting Path.

func (Path) AppendSegment 

func (p Path) AppendSegment(ps PathSegment) Path

AppendSegment is as per Join, but a shortcut when appending single segments.

func (Path) AppendSegmentInt added in v0.14.1 

func (p Path) AppendSegmentInt(ps int64) Path

AppendSegmentInt is as per AppendSegment, but a shortcut when the segment is an int.

func (Path) AppendSegmentString 

func (p Path) AppendSegmentString(ps string) Path

AppendSegmentString is as per AppendSegment, but a shortcut when the segment is a string.

func (Path) Join 

func (p Path) Join(p2 Path) Path

Join creates a new path composed of the concatenation of this and the given path's segments.

func (Path) Last 

func (p Path) Last() PathSegment

Last returns the trailing segment of the path.

func (Path) Len 

func (p Path) Len() int

Len returns the number of segments in this path.

Zero segments means the path refers to "the current node". One segment means it refers to a child of the current node; etc.

func (Path) Parent 

func (p Path) Parent() Path

Parent returns a path with the last of its segments popped off (or the zero path if it's already empty).

func (Path) Pop added in v0.17.0 

func (p Path) Pop() Path

Pop returns a path with all segments except the last.

func (Path) Segments 

func (p Path) Segments() []PathSegment

Segments returns a slice of the path segment strings.

It is not lawful to mutate nor append the returned slice.

func (Path) Shift 

func (p Path) Shift() (PathSegment, Path)

Shift returns the first segment of the path together with the remaining path after that first segment. If applied to a zero-length path, it returns an empty segment and the same zero-length path.

func (Path) String 

func (p Path) String() string

String representation of a Path is simply the join of each segment with '/'. It does not include a leading nor trailing slash.

This is a handy, but not a general-purpose nor spec-compliant (!), way to reduce a Path to a string. There is no escaping mechanism used by this function, and as a result, not all possible valid Path values (such as those with empty segments or with segments containing "/") can be encoded unambiguously. For Path values containing these problematic segments, ParsePath applied to the string returned from this function may return a nonequal Path value.

No escaping for unprintable characters is provided. No guarantee that the resulting string is UTF-8 nor NFC canonicalized is provided unless all the constituent PathSegment had those properties.

func (Path) Truncate 

func (p Path) Truncate(i int) Path

Truncate returns a path with only as many segments remaining as requested.

type PathSegment 

type PathSegment struct {
	// contains filtered or unexported fields
}

PathSegment can describe either a key in a map, or an index in a list.

Create a PathSegment via either ParsePathSegment, PathSegmentOfString, or PathSegmentOfInt; or, via one of the constructors of Path, which will implicitly create PathSegment internally. Using PathSegment's natural zero value directly is discouraged (it will act like ParsePathSegment("0"), which likely not what you'd expect).

Path segments are "stringly typed" -- they may be interpreted as either strings or ints depending on context. A path segment of "123" will be used as a string when traversing a node of map kind; and it will be converted to an integer when traversing a node of list kind. (If a path segment string cannot be parsed to an int when traversing a node of list kind, then traversal will error.) It is not possible to ask which kind (string or integer) a PathSegment is, because that is not defined -- this is *only* intepreted contextually.

Internally, PathSegment will store either a string or an integer, depending on how it was constructed, and will automatically convert to the other on request. (This means if two pieces of code communicate using PathSegment, one producing ints and the other expecting ints, then they will work together efficiently.) PathSegment in a Path produced by ParsePath generally have all strings internally, because there is no distinction possible when parsing a Path string (and attempting to pre-parse all strings into ints "just in case" would waste time in almost all cases).

Be cautious of attempting to use PathSegment as a map key! Due to the implementation detail of internal storage, it's possible for PathSegment values which are "equal" per PathSegment.Equal's definition to still be unequal in the eyes of golang's native maps. You should probably use the string values of the PathSegment as map keys. (This has the additional bonus of hitting a special fastpath that the golang built-in maps have specifically for plain string keys.)

func ParsePathSegment 

func ParsePathSegment(s string) PathSegment

ParsePathSegment parses a string into a PathSegment, handling any escaping if present. (Note: there is currently no escaping specified for PathSegments, so this is currently functionally equivalent to PathSegmentOfString.)

func PathSegmentOfInt 

func PathSegmentOfInt(i int64) PathSegment

PathSegmentOfString boxes an int into a PathSegment.

func PathSegmentOfString 

func PathSegmentOfString(s string) PathSegment

PathSegmentOfString boxes a string into a PathSegment. It does not attempt to parse any escaping; use ParsePathSegment for that.

func (PathSegment) Equals 

func (x PathSegment) Equals(o PathSegment) bool

Equals checks if two PathSegment values are equal.

Because PathSegment is "stringly typed", this comparison does not regard if one of the segments is stored as a string and one is stored as an int; if string values of two segments are equal, they are "equal" overall. In other words, `PathSegmentOfInt(2).Equals(PathSegmentOfString("2")) == true`! (You should still typically prefer this method over converting two segments to string and comparing those, because even though that may be functionally correct, this method will be faster if they're both ints internally.)

func (PathSegment) Index 

func (ps PathSegment) Index() (int64, error)

Index returns the PathSegment as an integer, or returns an error if the segment is a string that can't be parsed as an int.

func (PathSegment) String 

func (ps PathSegment) String() string

String returns the PathSegment as a string.

type UintNode added in v0.17.0 

type UintNode interface {
	Node

	// AsUint returns a uint64 representing the underlying integer if possible.
	// This may return an error if the Node represents a negative integer that
	// cannot be represented as a uint64.
	AsUint() (uint64, error)
}

UintNode is an optional interface that can be used to represent an Int node that provides access to the full uint64 range.

EXPERIMENTAL: this API is experimental and may be changed or removed in a future use. A future iteration may replace this with a BigInt interface to access a larger range of integers that may be enabled by alternative codecs.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.