helpers

Documentation

Index

• Variables
• type DagBuilderHelper
  • func (db *DagBuilderHelper) Add(node *UnixfsNode) (*dag.Node, error)
  • func (db *DagBuilderHelper) Close() error
  • func (db *DagBuilderHelper) Done() bool
  • func (db *DagBuilderHelper) FillNodeLayer(node *UnixfsNode) error
  • func (db *DagBuilderHelper) FillNodeWithData(node *UnixfsNode) error
  • func (db *DagBuilderHelper) GetDagServ() dag.DAGService
  • func (db *DagBuilderHelper) Maxlinks() int
  • func (db *DagBuilderHelper) Next() []byte
• type DagBuilderParams
  • func (dbp *DagBuilderParams) New(in <-chan []byte, errs <-chan error) *DagBuilderHelper
• type NodeCB
• type UnixfsNode
  • func NewUnixfsBlock() *UnixfsNode
  • func NewUnixfsNode() *UnixfsNode
  • func NewUnixfsNodeFromDag(nd *dag.Node) (*UnixfsNode, error)
  • func (n *UnixfsNode) AddChild(child *UnixfsNode, db *DagBuilderHelper) error
  • func (n *UnixfsNode) GetChild(ctx context.Context, i int, ds dag.DAGService) (*UnixfsNode, error)
  • func (n *UnixfsNode) GetDagNode() (*dag.Node, error)
  • func (n *UnixfsNode) NumChildren() int
  • func (n *UnixfsNode) RemoveChild(index int, dbh *DagBuilderHelper)
  • func (n *UnixfsNode) SetData(data []byte)

Variables

var BlockSizeLimit = 1048576 // 1 MB

BlockSizeLimit specifies the maximum size an imported block can have.

var DefaultLinksPerBlock = (roughLinkBlockSize / roughLinkSize)

DefaultLinksPerBlock governs how the importer decides how many links there will be per block. This calculation is based on expected distributions of:

• the expected distribution of block sizes
• the expected distribution of link sizes
• desired access speed

For now, we use:

var roughLinkBlockSize = 1 << 13 // 8KB
var roughLinkSize = 288          // sha256 + framing + name
var DefaultLinksPerBlock = (roughLinkBlockSize / roughLinkSize)

See calc_test.go

var ErrSizeLimitExceeded = fmt.Errorf("object size limit exceeded")

ErrSizeLimitExceeded signals that a block is larger than BlockSizeLimit.

Types

type DagBuilderHelper

type DagBuilderHelper struct {
	// contains filtered or unexported fields
}

DagBuilderHelper wraps together a bunch of objects needed to efficiently create unixfs dag trees

func (*DagBuilderHelper) Add

func (db *DagBuilderHelper) Add(node *UnixfsNode) (*dag.Node, error)

func (*DagBuilderHelper) Close

func (db *DagBuilderHelper) Close() error

func (*DagBuilderHelper) Done

func (db *DagBuilderHelper) Done() bool

Done returns whether or not we're done consuming the incoming data.

func (*DagBuilderHelper) FillNodeLayer

func (db *DagBuilderHelper) FillNodeLayer(node *UnixfsNode) error

FillNodeLayer will add datanodes as children to the give node until at most db.indirSize ndoes are added

warning: **children** pinned indirectly, but input node IS NOT pinned.

func (*DagBuilderHelper) FillNodeWithData

func (db *DagBuilderHelper) FillNodeWithData(node *UnixfsNode) error

func (*DagBuilderHelper) GetDagServ

func (db *DagBuilderHelper) GetDagServ() dag.DAGService

GetDagServ returns the dagservice object this Helper is using

func (*DagBuilderHelper) Maxlinks

func (db *DagBuilderHelper) Maxlinks() int

func (*DagBuilderHelper) Next

func (db *DagBuilderHelper) Next() []byte

Next returns the next chunk of data to be inserted into the dag if it returns nil, that signifies that the stream is at an end, and that the current building operation should finish

type DagBuilderParams

type DagBuilderParams struct {
	// Maximum number of links per intermediate node
	Maxlinks int

	// DAGService to write blocks to (required)
	Dagserv dag.DAGService

	// Callback for each block added
	NodeCB NodeCB
}

func (*DagBuilderParams) New

func (dbp *DagBuilderParams) New(in <-chan []byte, errs <-chan error) *DagBuilderHelper

Generate a new DagBuilderHelper from the given params, using 'in' as a data source

type NodeCB

type NodeCB func(node *dag.Node, last bool) error

NodeCB is callback function for dag generation the `last` flag signifies whether or not this is the last (top-most root) node being added. useful for things like only pinning the first node recursively.

type UnixfsNode

type UnixfsNode struct {
	// contains filtered or unexported fields
}

UnixfsNode is a struct created to aid in the generation of unixfs DAG trees

func NewUnixfsBlock

func NewUnixfsBlock() *UnixfsNode

NewUnixfsBlock creates a new Unixfs node to represent a raw data block

func NewUnixfsNode

func NewUnixfsNode() *UnixfsNode

NewUnixfsNode creates a new Unixfs node to represent a file

func NewUnixfsNodeFromDag

func NewUnixfsNodeFromDag(nd *dag.Node) (*UnixfsNode, error)

NewUnixfsNodeFromDag reconstructs a Unixfs node from a given dag node

func (*UnixfsNode) AddChild

func (n *UnixfsNode) AddChild(child *UnixfsNode, db *DagBuilderHelper) error

addChild will add the given UnixfsNode as a child of the receiver. the passed in DagBuilderHelper is used to store the child node an pin it locally so it doesnt get lost

func (*UnixfsNode) GetChild

func (n *UnixfsNode) GetChild(ctx context.Context, i int, ds dag.DAGService) (*UnixfsNode, error)

func (*UnixfsNode) GetDagNode

func (n *UnixfsNode) GetDagNode() (*dag.Node, error)

getDagNode fills out the proper formatting for the unixfs node inside of a DAG node and returns the dag node

func (*UnixfsNode) NumChildren

func (n *UnixfsNode) NumChildren() int

func (*UnixfsNode) RemoveChild

func (n *UnixfsNode) RemoveChild(index int, dbh *DagBuilderHelper)

Removes the child node at the given index

func (*UnixfsNode) SetData

func (n *UnixfsNode) SetData(data []byte)